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This book culminates a broad 
inquiry regarding what can be expected 
when questions are asked of mentally 
retarded persons. It started with 
a suggestion by the President's 
Committee on Mental Retardation that 
it might be useful to’ conduct a- 
periodic national polling of retarded 
citizens of all ages and circum-. 
stances as a means Of gathering 
input for policy making purposes. 


. The Committee had alreacy commis- 


sioned some work on defining the 
mentally retarded population and 
constituting a representative sample 
of that population. It then 
encouraged the Texas Tech University 
Research and Training Center in 
Mental Retardation to propose .a study 
in which the feasibility of inter- 
viewing ‘retarded persons would be 
explored. The study was intended 

to investigate the ability of 
retarded persons to answer questions 
and the reliability and validity 

of their answers; to compare their 
perspectives. with those of the 
nonretarded individuals (e.g., parents) 
who often speak for them; and to 
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obtain information about the needs, 
circumstances, and attitudes of 
retarded persons as well as guide- 
lines for constructing interview 
schedules in the future. 


The proposal was funded by the 
Rehabilitation Services Administra- 
tion with the idea that the Center 
would continue to interact with 

PCMR in charting directions. This 
indeed happened, and the President's 
Committee had a great deal of 
influence on the subject matter 

to be included in interview 
schedules as well as the kinds of 
research questions to be addressed. 
We know we. did not answer or even 
address all of the issues that 
concerned PCMR, but we’ are confident 
that this report will be of value 

to them as well as to a wider range 
of professionals and researchers in 
the developmental disabilities field. 


The authors have many others to thank. 
First, Gerard J,Bensberg, Director 

of the Research and Training Center, 
was indispensable in getting the 
project off the ground and supportive 
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throughout the study. Syng Nam Yough 
and Cheryl Smith contributed to data 
analysis. Staff in the following 
facilities and agencies deserve much 
thanks for their cooperation in making 
children and adults available to us 
for interviewing: the Lubbock State 
School, the Lubbock Public Schools, 
particularly the Special Education 
Department and the Ballenger School; 
Arkansas Mental Retardation-Develop- 
mental Disabilities Services, espe- 
cially the Arkansas Children's Colony 
at Conway; and the Austin Association 
for Retarded Citizens, especially the 
citizen advocacy program and its 
staff. Research of this nature 
always depends on the good will of 
agency and facility staff, from the 
top down, and in this project that 
good will was plentiful. Finally, we 
reserve special thanks for the parents 
and attendants, and most especially 
the retarded people, who actually 
participated ir our interviews. They 
knew, indeed they were explicitly 
told, that participating would not 


ii 


.selves. 


make a difference in their lives, 
would not help them with the problems 
that some of them faced. Yet they 
willingly gave it their best, often 
enjoying the chance to talk about 
themselves and to be listened to care- 
fully. All we can say by way of 
thanks is that we learned more than 
we ever hoped to learn about retarded 
people, as well as about how we: "non- 
retarded" people can communicate more 
effectively with them. Our hope is 
that this report, even though it 
uncovers many problems in getting 
meaningful answers from retarded | 
persons will encourage others to 
give them a chance to speak for them- 
We will show that good 
intentions are not enough, that soph- 
istication in phrasing ana structuring 
questions and skill in interacting 
with people are needed if the attempt 
is to be fruitful. But as the people 
we talked to have shown us, there is | 
much to be gained if we try as hard as 
they do to make the communication 
process work. 
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"If more people do not take a 
more active interest in the life of 
the mentally retarded then we will 
not have any place to-go except in 
an institutional living and that is 
not fair because we are God's ~“ 


creatures, to be like you are, 


considered as normal." Unsolicited 
letter cited by Baker, Seltzerand © 
Seltzer (1977). 


The mentally retarded adult who wrote 
these lines was acting as'an effec- 
tive advocate for community residen- 
tial alternatives to institutions 

and for the philosophy of normali- 
zation. As anyone who has worked 
with mentally retarded people knows, 
many of them tell us exactly what is 
on their minds (whether we want to 
know or not). Mentally retarded ‘peo- 
ple, like all people, have knowledge 
of what is going on in their lives, 
their likes and dislikes, and their 
hopes for thé future. Historically, 
however, their voices have not been 
heard. ,Certainly their perspectives 
have rarely been solicited by 
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professionals working with them or 
researchers studying them. To be 
sure, there are exceptions. Occasion- 
ally, first-hand accounts of the ex- 
periences of mentally retarded 
persons appear in print (see Stanovich 
and Stanovich, 1979, for a biblio- ` 
graphy). Occasionally "we" even 
solicit their opinions, as illustrat- 
ed by Nirje's (1972) report of a- 
Swedish conference of mentally retard- 
ed adults which yielded a set of 
provocative policy recommendations. 

To cite a notable example from the 


‘research literature, the deinstitu- 


tionalization study by Gollay, 
Freedman, Wyngaarden, and Kurtz (1978) 
not only involved: interviews with 
deinstitutionalized persons but also 
included a screening and critique of 


. proposed interview materials by a 


group of retarded consumers. None- 
theless, the predominant pattern has 
been for "us" to speak for "them," 
prompting Stanovich and Stanovich 
(1979) to advocate letting "them 
talk about us for a change" (p. 83). 


In part, neglect of the perspectives 
of mentally retarded consumers has 
been a result of the models of re- 
tardation that have guided the field. 
When mentally retarded persons are 
viewed as subhuman, as children, or 
as menaces to society, it simply 
follows that their opinions are 
irrelevant and that "we" are the 

ones who know what is and what should 
be happening in their lives. If, 
instead, we accept the concept that 
they are "God's creatures, to be _ 
like you are, consider: as normal," 
we are motivated to take them 
seriously as human beings with impor- 
tant things. to communicate. Precise- 
ly this concept is gradually infil- 
trating the field of mental retarda- 
tion today. The philosophy of 
normalization has had its impact, as 
has the broader movement fin society 
to guaraitee the civil rights of 
handicapped persons. Groups of 
handicapped persons, serving as their 
own advocates, have had a marked 
impact in seeing that these changes 
have come about. The mentally re-. 
tarded, though rarely speaking for 
themselves, have been represented 

by consumer Crganizations, parents, 
and other representatives. 


One of the major products.of the 
handicapped consumer movement has 
been the stipulation in a variety of 
legislation that handicapped clients 
or students be involved in making 
those decisions which affect them. 
Thus, rehabilitation agencies must 
insure that a client has agreed to. 
an individual rehabilitation plan, 
mental retardation facilities and 
agencies must involve clients in the 
development of habilitation plans, 
and, under appropriate circumstances, 
schools must involve not only parents 
but children themselves in the deve- 
lopment of individual education 
programs. These requirements make 

it mandatory that human services 
professionals begin to consider the 


perspectives and preferences of men- 
tally retarded persons. 


Service providers, counselors, and 


teachers are not the only ones who 
have a stake in obtaining information 
from mentally retarded consumers. 
Researchers, especially those study- 
ing deinstitutionalized and service 
delivery systems, depend on informa- 
tion gathered from mentally retarded 
persons, and indeed increasingly 
express the belief that such persons 
should be allowed to speak for them- 
selves, since it is their experiences 
which. are under study (e.g., Gollay, 
‘Freedman, Wyngaarden, & Kurtz, 1978). 
Virtually anyone who is involved with 
the mentally retarded has some need 
to understand the perspectives of 
mentally retarded persons and gather 
information from them. 


But how? How does one hear from 
people whose verbal skills, as a 
function of the very nature of mental 
retardation, are limited? It is with 
this question that the present report 
concerns itself. . 


The Nature of the Study 


j The research was inspired by the 
‘President's Committee on Mental Re- 
/ tardation. 


PCMR became intrigued by 
the idea of conducting a periodic 
poll of a representative -national 
sample of mentally retarded persons. 
The proposed survey, to be used with 
persons of all ages and living situ- 
ations, was viewed as a means of 
collecting informaticn about the 
circumstances, needs, and attitudes 
of mentally retarded individuals 
which cculd be used to shape national 
policy. The Research and Training 
Center in Mental Retardztion at Texas 
Tech was approached with the notion 
of conducting a study of the feasibi- 
lity of such a polling, and a pro- 
posal was then written and funded by 
the Rehabilitation Services 
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, such information. 


Administration. It became clear 
that PCMR was interested in a wide 
range of feasibility issues: how 
to coristruct a representative 


sample, how to access interviewees, & 


what to ask, how to ask it, and what 


to do with the résulting information. — 


In our own thinking about the proj- 
ect, we decided that the first and 
most important task was to determine 
what can be expected of mentally 


‘retarded persons in an interview 


situation; whether they can give 
answers, and whether their answers 
are reliable and valid sources of 
information about their circumstances, 
needs, and attitudes. If we assume 
that mentally retarded persons sho. ʻi 


be given opportunities to communicate . 
. a I 
it. / 


about their own lives and needs, 
is critical to explore the methodo- / 
logical issues involved in obtaining 


4 


In an important sense, then, the 


_ study became an exploration of the 
‘communication skills of the.mentally 


retarded; not their language deve- 
lopment, but their ability’ to under- 
stand questions and use speech to 
accurately convey facts and opinions. 
However, our intent was not just to 


explore deficiencies in communication 


skills among the mentally retarded. 
Comiunication is a two-way process, 
and the ability of mentally retarded 
persons to respond to a question 
might depend heavily on the form, 
clarity, and salience of the question. 
Thus we sought not only to understand 
the limits of retarded people's 
abilities to participate in inter- 
views but also to identify more and 
less effective ways of asking ques- 
tions of them. 


The four guiding questions of the 
project can be stated as follows: 


1. To what extent can retarded 
persons respond to questions 
in an appropriate fashion, 
and what factors affect 


their responsiveness? 


1. 


How reliable are such re- 

_ sponses, in the sense of 
being consistent over short 
periods of time? 


How valid are such responses, 
in the sense of being free 
of systematic biases and 
agreeing with infer iation 
provided by parents or care- 
takers or documented in 
records? 


What types of questions © 
appear to optimize respon- 
siveness, reliability, and 
validity? 


Organization of the Report. 

The body of this report is based 
on data obtained from five different 
interviewing studies. Each study 
involved interviewing a group of men- 
tally retarded children or adults 
‘about their circumstances, needs, 
and attitudes, and also interviewing 
a “significant other" such as a 
parent or attendant to obtain a 
second perspective on, each client. 

In Chapter 2, we briefly review the 
literature relevant to the project, 
considering both the literature on 
the communication skills of mentally 
retarded persons and the literature 
on the potential for biased responses 
in survey research with the general 
population. In Chapter 3, we de- 
scribe the design of our study,the 
samples interviewed, and the measure- 
‘ment procedures used so that this 
methodological information can be 
borne in mind in reading later chap- 
ters. In Chapter\4, the logistical 
challenges in conducting interviews 
are considered, and we offer sugges- 
tions based on our experiences for 
carrying out interviews with retarded 
persons. 


Because the five studies were very 

similar in basic purpose and design, 
we do not report the findings study 
by study. Instzad, the remainder of 
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‘the report is organized around the 


four guiding questions posed above. 
In Chapter 5, we draw from the 
various studies to examine respon- 
siveness, or the extent to which 
mentally retarded persons are able 
to answer questions. We describe 
the kinds of difficulties inter- 
viewees had in providing an appro~ 
priate answer; for example, in say- 
ing yes or no to a yes-no question 

or naming something in response to 

an open-ended question. We/look at 
responsiveness as an individual 
characteristic, describing how low 

in the IQ range one can go and ex- 
pect mentally retarded persons to 

be able to respond to Simple ques- 
tions, and we relate responsiveness 
to other factors such as the type © 
of question asked’ (yes-no versus 
either-or versus’ open-ended, and so 
on). . Starting with Chapter 6, we 
move beyond the issue of whether 

or not various questions elicit: 
appropriate answers and ask whether 
or not the responses obtained are 
useful. Chapter 6 focuses on the 
issue of reliability by analyzing 
data obtair:.4 in two of our studies 
in which institutionalized persons 
were asked the same questions twice, 
a week apart. If, in an interview- 
ing study, the answers given by a 
person change drastically from week 
to week, the information obtained is 
of little use. Again we examine 
factors (such as question format) that 
affect the extent to which answers 
are stable from week to week. In 
Chanter 7, we turn-to the validity 
issue, primarily by examining the 
extent to which answers given by rẹ- 
tarded persons and answers given by 
"significant others" agree. Although 


pi 


we do not assume that significant 


others represent "the whole truth," 
we are concerned with the extent to 
which pictures of a group provided 
by clients and/their parents or” 
attendants differ. , 


` 


a 


-response biases, 
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f 
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‘In Chapter 8, we describe direct 


a : i 
comparisons of alternative ways of 


requesting -the same information. 
Throughout the five studies, a number 


of different comparisons of this type 


were made. In some cases, we com- 
pared alternative question formats 
(e.g., yes-no questions versus either- 
or questions on the same topic), 
while in other cases we explored the 
effects of alternative question word-. 
ings or alternative probing tech- 
niques. In each case, we compared 
the questioning strategies at issue 
with reference to three criteria: 
responsiveness, absence of systematic 
and validity as 
measured by agreement with parents 

or attendants. Thus, we first asked 
which alternative yields appropriate 
responses from a higher proportion 


` of the population, assuming that the 


more useful questioning approach is 
the one to which most clients can re- 
spond. Then we compared the two al- 
ternatives in terms of the extent to 
which systematic biases, revealed in 
comparison of a client's responses 

to one format with his or her respon- 
ses to the other, invalidate respon- 
ses. Finally, we determined which 
question is associated with higher 
agreement with the responses given by 
parents or attendants, on the assump- 
tion that high agreement with 
significant others is an additional 
indicator of -validity of responses. 
Of any two approaches to soliciting 
the same information, then, the more 
useful of the two is the approach 
which optimizes responsiveness, does 
not appear to be associated with any 
systematic response biases, and yields 
answers that concur with those pro- 
vided by significant others. Chapter 
8 thus has many practical implications 
for anyone seeking improved methods 
of obtaining information from men- 
tally retarded persons. 


Finally, in Chapter 9. we attempt 
to summarize the major findings cf 
the study and their implications for 


1.4 


Q 
ERIC 


training mentally retarded persons 

in communication skills and for 
seeking information from them as a 
service provider, counselor, research- 
er, or simply interested party. Some 
readers may want to read Chapter 9 


tan 


first to get an overview of the study 
and its findings and then go back/to 
chapters of special interest. | 
Finally, we have provided a variety 

of appendix material of particular 

interest to those planning interview 
studies. 
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_ LESSONS 


As in any research project, we 
began our inquiry with reviews of 
previous research, hoping to find a 
body of knowledge on which to base 
our own study. We did not find a lit- 
erature on how best to ask questions 
As far as we 
can determine, ours is the first such 
study. However, we searched for 
clues in two related bodies of liter- 
ature; the literature on language and 
communication skills of the mentally 
retarded, and the literatnre on meth- 
odological issues in survey research 
with the general population, This 
chapter overviews what we found most 
pertinent to our research questions 
in these two very different bodies of 
literature. 


Information abcut the communication 
skills of mentally retarded persons 
is extensive, as deficits in lan-' 
guage and communication are part of 
the very nature of mental retardation. 


As experienced mental retardation re~ ~~. 


searchers, we of course knew that the 
limited verbal skills. of, many retard- 
ed persons would limit the, kinds of 
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FROM THE LITERATURE 


questions that could be asked of them. We 
knew that verbal interviewing would 
be all but impossible with profound- 
ly retarded persons, but were not as 
sure how much could be expected of 
those at higher levels of retarda- 
tion. We hoped to gather informa- 
tion about whether we could expect 

of a retarded person essentially what 
could be expected of a normal child 
of the same mental age, or whether 
some of the language and communica- 
tion difficulties of retarded per- 
sons are not so much a ‘problem of 
developmental delay as a matter of 
qualitative difference. For example, 
if retarded persons have been found 
to have serious difficulties in un- 
derstanding quantitative concepts . 

and terms, beyond those that. could be 
predicted on` the basi's’of ental age, 
we would want to avoid their use in 
interviews. 


Reviewing this vast literature was 
difficult, for participating in an 
interview is an activity that calls 
on all of a person's language capac- 
ities. Receptive skills are needed 
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>- to understand the question being 
asked; expressive skills to give an 


answer. Moreover, reception and ex- 
pression, or comprehension and repro- 
duc ion, must be applied to all of 
the ‘three subsystems of language: 
sound, meaning, or semantics, and 
gzammar or syntax. Thus, we had to 
be condemned with hearing deficits 
and speéch or articulatory problems 
among retarded persons; we had to ex- 
piore their vocabulary and comprehen- 
sion of various concepts; and we had 
to consider their ability to under- 
stand the structure of questions 
posed to them as well as to structure 
answers. X 


The selective lit@€rature review we 
present here considers in turn the 
sound, meaning, and ‘grammatical sys- 
tems of language and‘suggests what 
can be expected of retarded persons -.- 
in interview situations, However, 
this body of literature ‘did littie 

to tell us whether the answers of 
retarded persons can be expected to 
provide meaningful insight into their 
lives. We can ask a simple yes-no 
question, for example, and a retarded 
person may appear to understand. the 
question and answer "yes." But is 
that "yes" a reflection of reality? 
Would the person's parent or someone 
else who knows the person well con-‘. 
cur? These are issues which have not 


been addressed in the mental retarda- *Y 
tion field; yet. they are critical if 4 


we are concerned not just with lin- 
guistic competencies but with the use 
of those competencies for a purpose: 
to communicate accurately one's needs, 
circumstances, and attitudes. 


Fortunately, we had another body of 
literature to turn to for clues con- 
cerning the issue of response valid- 
ity. Researchers who use survey re- 
search methods with the general popula- 
tion have long been concerned with threats 
to response validity. They have repeat- 
edly found, forexample, that many people 


give biased answers to put themselves 

in a favorable light and that how one 
structures and phrases a question may 
influence what answers are given. We 
assumed that if certain biasing effects 
have been uncovered in survey research 
with the general population, there is 
no reason to believe that they will | 
not surface in interviews with re- 
tarded persons. On the contrary, as 
we will show, there is reason to be- 
lieve that threats to response vali- 
dity will be even more acute in inter- 
views of retarded persons: than they 
are in interviews of nonretarded per- 
sons. Thus thischapter's second sec- 
tion will overview what survey research- 
ers have discovered about response bias. 


Language and Communication 


Among Retarded Persons 

Speech and communication problems 
are generally viewed as characteris- 
tic of mentally retarded persons 
(Blount, 1968; Karlin & Strazzula, 
1952; Keane, 1972), and have frequent- 
ly been used as important diagnostic 
indicators of mental retardation 
(Blount, 1969; Matthews, 1971). Un- 
fortunately, research has provided 
little consistent normative data on 
the general level of linguistic com- 
petence to be expected in retarded 
persons of a particular intelligence 
level (Matthews, 1971). This lack of 
eneral expectations about language 
Qmpetencies can be attributed to the 
varied approaches used to study lan- 
guage development in the retarded 
(Yoder & Miller, 1972). Studies are. 
difficult to compare directly due to 
methodological differences, variations 
in the populations studied, and dif- 
ferences in the specific types of lan- 
guage behavior measured. However, 
those who have surveyed theliterature 
of language development of the mentally 
retarded have been able to draw some 
broad conclusions. 


LANGUAGE AND INTELLECTUAL 
FUNCTIONING 

The retarded child typically ac- 
quires language and speech consid- 
erably later than the nonretarded 
child (Baroff, 1974; Cromer, 1974; 
Matthews, 1971; Yoder & Miller, 
1972). While many developmental 
sequences are delayed for retarded 
children, the areas of speech and 
higher intellectual functions show 
the greatest amount of delay (Karlin 
& Strazzula, 1952). For example, 
. Schlanger (1953a) reported that the 
‘average age of onset of speech in a 
group of institutionalized retarded 
children was slightly over three 
years, while nonretarded children 
begin speaking about the first year 
(McCarthy, 1954). However, even 
though retarded children are reported 
to lag behind their nonretarded peers 
on such measures as sentence length, 
sentence complexity, discrimination 
of speech sounds, and percentage of. 
nouns used (Spreen, 1965), most 
authors agree that the course of- 
language development generally 
parallels that of normal children. 
That is, it is not qualitatively ' 
different (Lackner, 1976; Lenneberg, 
Nichols, & Rosenberg, 1964; Yoder & 
Miller, 1972). 


‘Baroff (1974) has attempted to pro- 
vide some ‘rough expectations re- 
garding the lingyistic competence 

of retarded children and adults as 

a function of mental age and degree 
of retardation. As shown in the sum- 
mary Of these norms (Table 2.1), 
Baroff suggests that the major 
language deficit for retarded persons 
involves expressive language abili- 
ties. As has been found true of 
normal children, retarded persons 

are usually able to understand more 
speech than they are eble to produce. 
Therefore, a general strategy for 
interviewing the retarded might be 

to devise questions which minimize 
productive demands (e.g., through 
simple yes-no questions, or forced- 


choice questions, or even questions 
that require only pointing). 


Baroff's norms for retarded children 
also suggest that the order of lan- 
guage acquisition follows the devel- 
opmental sequence reported for nonre- 
tarded children (McCarthy, 1965). 

For our purposes, the most useful 
information that emerges from 
Baroff's data is that the ability to 
respond to questions is not evident 
prior to mental age four or five. 
Therefore, efforts directed at inter- 
viewing retarded persons below this 
mental age may be inadvisable and of 
limited value. ` 


Baroff also emphasizes that speech 
problems and language delays are more 
pronounced at the lower levels of in- 
tellectual functioning, and that 
speech development in the retarded 
child is closely associated with men- 
tal age. For the most part, research 
on the language development of the 
retarded supports these observations 
(Karlin & Strazzula, 1952; Lyle, 
1961b; Sirkin & Lyons, 1943). 


As a general rule, then, the higher 
the level of intelligence, the better 
the language ability of the retarded 
individual (Keane, 1972). Many early 
studies indicated that speech and 
language measures are positively cor- 
related with intellectual level in 
retarded populations, even though 
these correlations are often of only 
a moderate degree. For example, 
Spradlin (1963) reviewed four studies 
which correlated articulation and 
intelligence measures for institu- 
tional populations and found that 

the coefficients ranged from .41 to 
.58. Schlanger (1958) found that 
auditory word discrimination corre- 
lated .59 with mental age, and other 
studies have reported correlations 
ranging from .42 to .76 between vo- 
cabulary size and intelligence 
(Condell, 1959; Mein, 1962; Sloan & 
Bensberg, 1954). By contrast, 
chronological age has little or no 
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Profound 


Receptive: 


Mild Expressive: 
Moderate 
Severe 
Profound 
Receptive: 


Mild Expressive: 
Moderate 
Severe 
Profound 
Receptive: 


Mild . Expressive: 
Moderate 
Severe 


Mild Expressive: 
Moderate 
Mild Expressive: 


2.4. 


Language Behavior 


Expressive: 


Severe Expressive: 


bility. At more severe levels of 


conversation and use complex sen- 


moderately retarded children). 


Essentially normal fluency though 
Pronunciation problems persist. 


age, chronological age, and degree of retardation from Baroff (1974) 

















Imitates sounds. Laughs or smiles 
in response. May say "mama" or 
"dada". At a pre-speech level: 
crying, vocalization, and gestures 


Some undexstanding of gestures and 
very familiar words (e.g., "no"). 


May use one or two words, but pre- 
speech forms continue to predomi- 
nate. 


May have considerable speaking 
vocabulary though speech will be 
particularly impaired in children 
with more than mild retardation. 
If nonverbal, as in severely and 
profoundly retarded; there may be 
use of gestures to communicate. 


Understand simple verbal communi- 
cations, (e.g., following direc- 
tions, responding to questions). 


May have considerable speaking 
vocabulary though speech will be 
particularly impaired in children 
with more than mild retardation. 
If nonverbal, as in severely and 
profoundly retarded, there may be 
use of gestures to communicate. 


Understands simple verbal commun i= 
cations, (e.g., following direc- 
tions, responding to questions). 


Mildly retarded child may be using 
fairly normal sentence structure 
and have speech of good intelligi- 


retardation, language may be at 
the phrase or single-word level and 
alse be indistinct in pronunciation. 


May be able to carry on a simple 


tences (more true of mildly thau 
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Table 2.1: continued 













Degree of Language Behavior 


Retardation 


Mental 
Age 













Language is adequate for ordinary purpose of 
communication at both expressive and receptive 
levels. 


Mild 








Language is generally functional for purposes 
of communication, but intelligibility of spesch 
may be much impaired. 





Moderate 







4-6 yrs Severe Understanding language ie likely to be much 
better than the ability tc express it. Speech 
may be very poorly articulated and difficult to 
understand. 

0-4 ` yrs Profound Ability to understand is likely to far exceed 





ability to speak. There may be little or no 
speech. Capable of following simple directions. 
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relationship to linguistic ability 

in the retarded child (Bangs, 1942; 
Schlanger, 1953b). The relationship 
between intelligence and language 
generally holds for persons with a 
mental age of eight years or less, 

at which point the degree of corre- 
lation diminished (Spreen, 1965). 

It is interesting to note that Goda 
and Griffith (1962) have reported 
similar results for nonretarded 
children. They contend that speech 
development is directly related to 
intellectual growth in normal chil- 
dren ` | age eight. After this 
point, further development is minimal 
and consists mainly of mastering more 
complex sentence structures and 
gradually increasing vocabulary size. 
Therefore, prior to mental age eight, 
one would expect to find language 
acquisition and proficiency closely 
related to intellectual level in both 
retarded and nonretarded populations. 
The language development of the re- 
tarded child would be very similar 


‘to that of the nonretarded’ child of- 


an equivalent mental age. Conse- 
quently, retarded persons below men- 
tal age eight would not have mastered 
the fundamental aspects of language 
and would evidence more immature 
speech forms than nonretarded persons 
of the same chronological age. 


While most authors ‘report that speech 
impairments and language deficits are 
more numerous and severe in retarded 
populations, it is generally accepted 
that no specific types of language 


disorders are found in the speech of . 


the retarded that are not also found 
in nonretarded populations (Keane, 
1972; Matthews, 1971; Spreen, 1965). 
Similarly, Lenneberg and his asso- 
ciates (1967; Lenneberg, Nichols, & 
Rosenberg, 1964) have concluded that 
no structures are found in the lan- 
guag2 of the mentally retarded that 
do not occur in the course of devel- 
opment for nonretarded children. 
These authors contend that differ- 
ences observed in the speech and 


language patterns of the retarded are 
primarily quantitative in nature and 
do not represent qualitative language 
differences. 


In contrast to previous research, how- 
ever, Lenneberg et al. (1964) reported 
that chronological age was a better . 
predictor of language achievement than 
intelligence in a three-year study of 
Down's Syndrome children whose ages 
ranged from 3 to 22. They also con- 
cluded that language acquisition in 
Down's Syndrome children depends more 
on physiological maturation than on 
cognitive development. Lenneberg's 
findings highlight two important con- 
siderations. First, etiology plays a 
role in determining the course of 
speech development and competence in 
the retarded (Cromer, 1974; Spreen, 
1965; Webb & Kinde, 1967). Therefore, 
it would seem necessary to distinguish 
type of mental retardation as well as 
degree of retardation when assessing 
the linguistic ability of the re- 


tarded. Secondly, research has sug- 


gested that factors other than cog- 
nitive ability may influence language 
acquisition, and therefore we must 
recognize how such factors affect the 
language behavior of the retarded. 


LANGUAGE AND ETIOLOGY 
Early research designed to identify © 
specific language deficits among the 
various diagnostic categories of men- 
tal retardation has typically compared 


- Down's Syndrome or: brain-damaged 


groups with other groups of mentally 
retarded persons. In jeneral, brain- 
damaged retarded children show the 
greatest amount of unevennessin their 
expressive and receptive language 
abilities (Rigrodsky & Gou 1951). 
For example, when compared to non- 
brain-damaged groups, they are inferi- 
or on auditory discrimination and 
perception tasks (Spreen, 1965; 
Schlanger, 1958), and are inferior in 
making verbal associations, integrat- 
ing verbal concepts, and using 
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visual-motor perceptual abilities in 
speech production (Gallagher, 1957). 
On the other hand, the brain-damaged 
as a group typically have larger vo- 


- cabularies, use more advanced modes 


of verbal expression, Jive more de- 
tailed and complete definitions for 
words (Bijou & Werner, 1944, 1945), 
and are better on verbal imitation 
and speech production tasks 
(Gallagher, 1957). 


In contrast, Down's Syndrome children 
as a whole are consistently poorer on 
most language measures when compared 
to other groups of retarded children 
(Lyle, 1960c, 196la, b). Mein -(1961) 
has noted that Down's Syndrome chil- 
dren decrease their use of nouns at 
a later age than do most other groups 
of retarded children, which has been 


‘interpreted as representing a more 


concrete and immature style of think- 
ing. Speech defects are also reported 
to occur’more frequently among Down's 
Syndrome children (Schlanger & 
Gottsleben, 1957). In particular, 
several authors have noted a greater 
incidence of stuttering among Down's 
Syndrome groups (Gottsleben, 1955; 
Schaeffer & Shearer, 1968; Schlanger, 
1953a). Finally, Bilovsky and Share 
(1965) suggest that a deficit in 
auditory-sequential memory is a major 
factor in the language rétardation of 
Down's Syndrome children. 


Many of these findings have been sup- 
ported and extended in a recent inves- 
tigation by Rohr and Burr (1978). 

They compared four diagnostic groups 
of retarded subjects (Down's Synarome, 
biological brain damaged, environmen- 
tally caused, and unknown etiology) 
according to their performance on the 
Illinois Test of Psycholinguistic 
Abilities. Ali four groups exhibited 
a similar pattern of scores on the 
subtests, doing best in verbal expres- 
sion and worst in auditory memory. | 
However, it was found that the Down's 
Syndrome group consistently scored 
lower than the other three groups on 


all auditory-verbal subtests except 
verbal expression. In contrast, the 
environmentally-caused retardation ~ 
group scored higher on all subtests 
than did the other diagnostic groups, 
although the differences were not 
always significant. The scores for 
the other two diagnostic groups were 
more variable, but generally fell be- 
tween the extremes defined by the 
Down's Syndrome group and the environ- 
mentally-caused retarded group. The 
authors suggest that an auditory mem- 
ory deficit contributes to the lan- 
guage problems of all retarded groups, 
with Down's Syndrome children simply 
being affected the most rather than 
being unique as Bilovsky and Share 
(1965) implied. Therefore, etiology 
appears to have an overall influence 
on the degree of auditory-verbal de- 
ficienciesin the mentally retarded, 
but no specific pattern of language 
disability can be linked to a epegerte 
type of retardation. 


LANGUAGE AND OTHER FACTORS 

Matthews (1971) suggests that there 
may be many other explanations for 
poor speech development and language 
competence besides low intellectual 
functioning. For example, emotional 
disturbances, hearing loss, and lack 
of verbal stimulation may delay lan- 
guage acquisition or result in abnor- 
mal speech patterns. 


Environmental deprivation is frequent- 
ly cited as producing negative effects 
on the language development of chil- 


dren, and the adverse effects of in- 


stitutionalization on language devel- 
opment has been studied extensively 

in both retarded and nonretarded pop- 
ulations (Spradlin, 1963; Spreen, 1965; 
Spitz, 1945, 1946; Skeels', Udegraff, 
Wellman, & Williams, 1938). Institu- 
tional living has been associated with 
lower scores on vocabulary measures 

in both normal (Skeels et al. 1938) and 
retarded children (Little & Williams, 
1937). Schlanger (1954) reported 
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that institutionalized retarded 
children produce shorter sentences, 
while Sievers and Essa (1961) found 
that institutionalized children 
scored lower on the Developmental 
Language Facilities Test than did 

a matched group of nonins*itution- 
alized children. Finally, Lyle 
(1959, 1960a, b, 1961b) has con- 


cluded from a series of studies on ` 
_ the effects of institutionaliza- 


tion that noninstitutionalized re- 
tarded children are superior on 
overall language ability and that 
the negative effects of institu- 
tionalization may be greater at 
certain points in the language 
development sequence. These 
authors point to the effects of 
family separation, emotional de- 
privation, lack of good speech - 


models, and lack of speech moti- 
"yation as the causes of these def- 


icits. 


‘More recent research has sought to 


determine if some language func- 
tions are more affected by insti- 
tutionalization than others are. 
For example, Montegue, Hutchinson, 
and Matson (1975) analyzed psycho- 
locical and sociological language 
content categories and found little 
difference between language samples 


of. institutionalized and noninsti- 


tutionalized retarded children on 
these measures. They suggest that 
institutionalization may produce 
the same negative effects on lan- 
guage content as were reported pre- 
viously for measures of language 
structure and vecabulary. 


More recently, McNutt and Leri 
(19:9) examined both the structure 
of spoken language and specific 


- auditory-verbal abilities as meas- 


ured by the Illinois Test of ` 
Psycholinguistic Abilities among 
institutionalized and noninsti-~ 
tutionalized retarded children. 


Their findings supported the 
suggestion of Montegue et al. (1975) 
that institutionalization may have 
selective effects on the language 
abilities of the retarded. In 
particular, they found that their 
institutionalized groups had sig- 
nificantly poorer scores on audi- 
tory reception, verbal expression, 
and auditory closure. However, in 


gontrast to previous research, they 


¢ 
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found no differences between the 
two groups on measures of language 
structure. Overall, then, when 
differences are found between groups 
of ‘institutionalized and noninsti- 
tutionalized retarded persons, they 
generally favor those who do not 
live in institutions, but more 
markedly in some areas of competence 
than in others. 


In addition to examining the effects 
of institutionalization, current re- 
search has also looked at differences 
in the early mother-child verbal 
interactions of retarded and non- 
retarded infants. `-vbal interac- 
tions between retarccu children 

and their mothers appear to be 
different from the communication 
patterns of nonretarded children and 
their mothers (Buium, Rynders, & 
Turnure, 1974; Gutmann & Rondal, 
1979; Marshall, Hegrenes, & Goldstein, 
1973). For example, Buium et al. 
(1974) found that mothers-of Down's 
Syndrome children appear to be 
modeling shorter and grammatically 


less ‘complex verbalizations for 


their retarded children than mothers 
of nonretarded children do. Current- 
ly, we do not know whether such 
verbal modifications made by the 
mothers of retarded children ad-. 
versely affect their children's 
language development. Gutmann and 
Rondal (1979) have suggested that 
these adaptations may not be func- 
tionally inappropriate to the re- 
tarded child's language comprehension 
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needs; however, they may contribute 
to the simplé language structure 
generally reported to occur in the 
speech of the mentally retarded. 


In summary, then, speech and com- 
munication problems abound ‘among 
retarded persons, with the major 
difficulties being evident in the 
expressive area. Even though 
language development appears to be 


_ delayed in the majority of re- 


tarded children, the sequence of 
development has been found ‘to 
parallel that of nonretarded chil- - 
dren and is not considered to be 
qualitatively different. A lack of 
general language expectations pre-: 
vents us from making accurate pre- 
dictions as to what can be expect:ed 
of the retarded person of a par- 
ticular mental age; however, on the 
average, language competence in- 
creases and speech problems de- 
crease with higher intellectual 
functioning. In addition, a number 
of other factors such as environ- 
ment may contribute to the lan- 
guage problems of the retarded and 


produce more severe language def- 


icits than would be expected from 
their level of intellectual func- 
tioning. We will now focus on 
more specific speech and language 
deficits among the retarded that 
should be taken into account in 
interviewing them. 


PROBLEMS. RELATED TO SPEECH AND 
HERRING DEFECTS 

Research on the prevalence of 
speech and hearing disorders among 
the mentally retarded has been re- 
viewed extensively by several 
authors (Bensberg & Sigelman, 1976; 
Keane, 1972; Matthews, 1971; 


" Spradlin, 1963; Spreen, 1965). 


There is substantial evidence that 
speech and hearing defects are more 
frequent in mentally retarded pop- 
ulations than in the general pop- 


- ulation. 


However, wide variation 
among prevalence figures is also 
evident. i 


Speech Defects 
Speech defects among retarded 
interviewees may make it difficult 


. for them to express themselves and 


for interviewers tọ understand them. 
Spradlin (1963) reviewed 14 studies 
which surveyed the prevalence of 
speech defects in mentally retarded 
children and ccncluded that the per- ' 
centage of institutionalized re- 
tarded persons having speech defects 
varies from 57 to 72 percent.. 
Matthews’ (1971) review of the lit- 
erature reported figures which varied 
from 18 to 94 percent, with median 
rate.of 56 percent. Thirteen of the 
18 retarded samples studied were also 
from institutional settings. 


According to Spreen (1965), the pre- 
valence of speech defects varies as 

a function of intelligence within 

the retarded population. He esti- 
mates that almost 45 percent of those 
classified as mildly retarded exper- 
ience speech problems of some kind. 
This figure increases to 90 percent 
in the moderately retarded range, 


` and nearly all children in the 


severely and profoundly retarded 
ranges demonstrate some kind of 
communication disorder, which may 
include lack of speech. 


Those authors who have attempted to 
assess.the frequency with which 
specific speech problems occur among 
the mentally retarded tend to agree 
that articulation and voice disorders 
constitute the largest percentage 

of speech difficulties (Keane, 1972; 


` Spradlin, 1963). -There is evidence 


to indicate that this is also true 

in nonretarded populations (Spradlin, 
1963) and.that, in general, the types 
of speech problems found in retarded 
groups are also found in normal 
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populations (Keane, 1972; sprad- 
lin, 1963; Spreen, 1965). 
Stuttering is another speech diffi- 
culty which, like articulation and. 
voice disorders, seems to be es- 
pecially common among the retarded. 
Keane (1972) rates stuttering as the 
third most common speech defect of 
the mentally retarded, but noted 
that the higher frequency of 


ee eis ing in this group is still 
controveysial. For example, Karlin 


and Strdzzula (1952) observed only 
one case of stuttering out of 50 
mentally retarded children, a find- 
ing which corresponds with the 
prevalence figures typically cited 
for the general population (2%). 
There seems to be more agreement 
that a higher. than normal incidence 
of stuttering exists among persons 
with Down's Syndrome (Shubert, 1966); 
Schlanger & Gottsleben, 1957; 
Gottsleben, 1955). In addition, 
voice disorders such as low husky 
voices are reported to be especially 
common among Down's Syndrome: children 
(Benda, 1946; Karlin & Strazzula, 
1952; West, Kennedy, & Carr, 1947), 


Mentally retarded persons have also 
demonstrated difficulty in making 
sound discriminations, which con- 
tributes to poor speech production. 
In one study, Schlanger (1953b) de- 
termined that only 19 percent of his 
sample of retarded children ages 8 
to 16 years were able to make sound 
discriminations at the second grade 
level of proficiency, even though 
the average mental age for the sample 
was 6-8. In a previous investiga- 
tion, Schlanger (1953a) reported 
that these same children were also 
inferior to nonretarded children 
matched on mental age in auditory 
memory span for vowels .. 


The development of speech sounds 
and speech proficiency in retarded | 
children has been examined in sev- 


eral studies. Matthews (1971) cites 
a 1942 study by Bangs as one of the 
most systematic studies of typical 
errors found in the speech of the 
mentally retarded. Bangs.analyzed 
the types of verbal substitutions, 
omissions, and additions made by 53 
retarded children. He found that 
mental age was more closely related 
to speech proficiency as measured by 
the number of articulation errors 


(r = .41) than either intelligence 
(r = .33) or chronologivzal age 
(r = -.11). In general, the types 


of articulatory errors made by re- 
tarded children were very similar 
to those produced by nonretarded 
children of a comparable mental age. 


During thè same year, Irwin (1942) 
reported on the speech sound devel- 
opment of ten mentally retarded chil~ 
dren ranging from one to five vears 
old over a one-year period. Unlike 
Bangs, he concluded that speech 
sound development in his sample of 
retarded children was qualitatively 
different from that of normal chil- 
dren, because their level of pro- 
ficiency approximated that of nor- 
mal children less than one-year old. 
However, Irwin's findings may acm 
tuslly indicate that speech sound’ 
acquisition was severely delayed 

in this group’ rather than being 
different from that reported for 
normal children because his sample 
was’ still quite young. 


A more extensive study was reported 
by Karlin and Strazzula (1952), 
using subjects with a wider range of 
intelligence scores and ages than 
those studied by Irwin. The authors 
observed that the majority of chil- 
dren had speech defects which were 
similar to those found in normal 
children, but that the defects were 
more prevalent and often more severe. 
Similarly, Schlanger's (1953a, 1953b) 
data.also point to a delay of normal 
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speech development in retarded 
children, rather than the develop- 
ment of deviant speech patterns, and 
support Bangs' rather than Irwin's 
findings. Finally, Goda and Griffith 
(1962) found that both intelligence 
and mental age were related to 
articulation scores in a sample of 
retarded adolescents (r = .58 and 
-52, respectively). 


Therefore, when interviewing the 
mentally retarded one would expect 
to find the same types of speech 
defects commonly found in normal 
children, but with greater frequency 
and often greater severity. The 
evidence seems to suggest that 
speech proficiency is related to the 
retarded individual's Level of , 
intelligence, and one could expect 
the speech capacity of the retarded - 
person to correspond to that of a 
nonretarded child of a comparable 
mental age. Several authors have 
indicated that articulation and 
voice disorders are the most common 
types of speech difficulties among 
the retarded. This would suggest , 
that interview techniques which 

make minimal expressive demands on 
the interviewee would be most ' 
effective with mentally retarded 
persons. 


Hearing Disorders 

Hearing disorders not only _ 
interfere with language development 
but obviously make for difficulties 
in understanding questions asked 
in an interview. Like the liter- 
ature on speech disorders, the 
literature on hearing disorders. , 
among the mentally retarded suggests 
wide variation among the estimated 
percentages of retarded persons i 
with hearing impairment. However, 
the estimates have consistently 
exceeded the five percent pre- 
valence figure commonly cited for 
hearing losses in normal public 


eof 


school children. Individual 

studies have reported estimated per- 
centages of hearing problems in 
retarded populations ranging from 
two-and-a-half to eighteen times 
that found in the general population 
(Birch & Matthews, 1951; Rittmanic, 
1959; Schlanger & Gottselben, 1956). 
Therefore, despite conflicting 
figures, the consensus is 

that the «ccurxence of hearing 
problems is considerably greater 

in this population tan in nonre- 
tarded populations (Bensberg & 
Sigelman, 1976; Matthews, 1971; Lloyd 
& Reid, 1967). 


One major. consideration in deter- 
mining, the prevalence of hearing 
disorders in any population involves 
the choice of criteria for defining 
a hearing loss. Several authors 
have noted that variation in hearing 
loss criteria has created problems 
in comparing prevalence figures 
(Webb & Kinde, 1967; Kodman, 1958; 
Lloyd & Reid, 1967). Kodman (1958) 
reviewed seven studies on the pre- 
valenceof hearing problems in the 
retarded and found that the rate of 
hearing impairments ranged from 13 
to 49 percent. He proposed that the 
criteria for defining: a hearing prob- 
: lem should be standardized to re- 
present a loss of 30 decibels or 
more in either ear At one or more 
frequency (range 125 to 8000 cycles 
per second), on thé basis that most 
hearing losses below this level are 
not severe enous [to interfere with 
speech reception and language acqui- 
sition. According to these criteria, 
he estimated that the prevalence of 
hearing tosses Among the mentally 
retarded would/then average 15 to 


20 percent. While Lloyd (1970) 


" agreed with these estimates, he 


argued that’a less stringent crite- 
rion for hearing loss is more appro- 
priate. 
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Most studies which have examined the 
severity of hearing disorders at 
various levels of retardation have 
found almost no relationship between 
the percentages of hearing probiems 
observed and the level of measured 
intelligence (Reynolds & Reynolds, 
1979; Rigrodsky, Prunty, & Glovsky, 
1961; Schlanger & Gottsleben, 1956; 
and Siegenthaler & Krzywicki, 1959). 
However, this may be an artifact, 
since severely and profoundly re- 
tarded persons are often untestable 
by audiometric screening (Hogan, 
1973). In addition, hearing dis- 
orders are known to increase with 
age in all populations, and there 
are several studies which demon- 
strate this fact in retarded pop- 
ulations (Kodman, 1958; Kodman, 
Powers, Philip, & Weller, 1958; 
Schlanger & Gottsleben, 1956). 
Schlanger and Gottsleben (1956) 
observed a 25.7 percent hearing loss 
rate in their under 20 age group, 
but this percentage increased to 
41.4 percent in their over 20 age 
group. Kodman et al. (1958) used 
the same hearing loss criteria and 
obtained more conservative figures. 
Their young group (mean age = 15.4 
years) demonstrated a 19.0 percent 


- rate of hearing loss, while their 


older group (mean age = 38.7 years) 
showed a 23.8 percent hearing loss 
rate. 


Even though there is ccnsiderable 
variation among the surveys of hear- 


` ing problems in the retarded, on the 


average, hearing impairments seem to 
occur four to five times more often 
in retarded children than, in normal 
children. Studies which use less 
stringent criteria to define a hear- 
ing loss and which evaluate older 


subjects frequently show higher prev- 


alence figures. .Finally,. although 
there is no solid research evidence, 
one must. consider the possibility 
that a hearing loss in a retarded 
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child is more handicapping than the 


same loss in a normal child. Lloyd 
and Cox (1972) have suggested that 
"the retarded do not demonstrate the 


- same adaptive behavior or capacity 


to compensate for sensory involve- 
ment such as auditory or visual 
losses as do norretarded individuals" 
(p. 22). Possibly then, a hearing 
handicap of even minor severity coulda 
represent a serious disability to the 
retarded person and decrease his or 
her.ability to learn about the world 
and communicate with others. 


The Semantic System of the Mentally 
Retarded 

Semantics or meaning in the lan- 
guage of retarded persons has impli- 
cations for how well they can under- 
stand the words, concepts, and ques- 
tions put to them. Studies which 
compare the vocabularies and lexical 


usage of retarded children and non- 


retarded children focus primarily x 
on such things as vocabulary size 
and diversity, degree of abstraction 
in word usage, word frequencies, and 


' the relative usage of various parts 


of speech. In general, most studies 
indicate that retarded children use, 
significantly more concrete word 
forms and references than do normal’ 
children of a comparable mental age’ 
(Yoder & Miller, 1972).. For example, 
Sloan and Cutts (1947) determined 
that the essential difference between 
"easy" and "difficult" items on the 
Stanford-Binet Intelligence Scale for 
retarded children was the degree of 
abstraction reqyired to pass an item. 


Several early studies examined the re- 


<- tarded person's lability to formulate 


abstract word definitions. Papinia 
(1954) demonstr ted that as mental 
age increased; the ability to define 
words in more | stract terms also 


_improved for retarded persons. At 


any given mental age level retarded 
children could acceptably define as 





many words as nonretarded children 
but their definitions were often 
simpler and more concrete. These 
findings suggest that retarded chil- 
dren do not differ qualitatively 
from normal children in terms of de- 
fining words, but merely develop the 
ability to provide more abstract de- 
finitions later. 


Badt (1958) has suggested that length 
of institutionalization may be a sig- 
nificant factor influencing the level 
of abstraction exhibited by retarded 
persons. Abstracticn scores for a 
group of moderately and mildly re- 
tarded state school residents were 
negatively associated with the num- 
ber; of years each child had spent in 
the institution (r = -.61), while 
chronological age and mental age 
were only modestly related to level 
of abstraction (x = .34 and .24, re- 
spectively). Bact concluded that in- 
stitutional living interferes with 
the retarded child's ability to form- 
ulate abstract relationships and to 
manipulate concepts. 


The inability of retarded children to 
express abstractions is also seen} in 
their performance on tests which [call 
for identifying similarities among 
different objects. Retarded children 
frequently insist on naming differ- 
ences rather than abstracting a tom- 
mon feature from a group of items. 
For example, Griffith and Spitz (1958) 
found that retarded boys had trouble 
abstracting a common characteristic 
from a group of words similar on one 
dimension, even though they could 
adequately define the words individ- 
ually. w f 


Therefore, the evidence seems ;to in- 
dicate that retarded children jare 
less apt than normal chilaren/ to use 
abstract concepts. In addition, 
there is some evidence to suggest 
that this ability to use language at 
a more abstract level becc:zes more 


impaired the lonaer a child remains 
institutionalized. This line of ‘ce- 
search, even though it deals with 
production rather than comprehension, 
suggests that retarded interviewees 
may have special difficulty under- 
standing questions which contain ab- 
stract words or concepts. 


The size and composition of an indi- 
vidual's vocabulary is another im- 
portant aspect of semantic develop- 
ment. Mein and O'Connor (1960) com- 
pared the oral vocabularies of in- 
stitutionalized retarded persons 
with those of nonretarded children 
of.a comparable mental age. Re- 
tarded persons were found to average | 
neariy one-third more words in their 
individual vocabularies than nonre- 
tarded children. Spreen (1965) re- 
viewed several studies confirming 
this observation that mentally re- 
tarded persons tend to have larger . 
vocabu.2zries than nonretarded chil- 
dren of the same mental age. He 
attributes their larger vocabulary 
size to the fact that mentally re- 
tarded subjects are typically older 
and have had more exposure to lan- 
guage than their nonretarded coun- 
terparts. 


Looking at the kinds of words in the 
vocabularies of retarded and normal 
children, Mein and O'Connor found 
that institutionalized retarded 
persons share more words in common 
than do normal children of the samne 
mental age, possibly because insti- 
tutionalization causes them to have 
more experiences in common with 
their peers. Nonretarded children 
exhibited greater individuality in 
their vocabulary, despite the fact 
that their total vocabulary size was 
smaller. It should be noted that 
in this study the size of a retard- 
ed individual's vocabulary did ` ` 
not seem to be affected by the 
length of institutionalization 
(r = .17) but was significantly 
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related to the retarded person's 


mental age (r=. 72) i 


In a study comparing word frecuecncy 
and word usage of state school: resi- 
dents and nonretarded children, Beir, 
Starkweather, and Lambert -(1969) ob- 
served only minor differences in the 
types of words used by retarded and 
nonretarded children. For example, 
retarded children tended to use posi- 
tive words like "yes" and "O.K." more 
often (see the discussion of jacqui- 
escence that follows). 
ren also used a greater numbe:’ of 
self-references (me, I, my, etc.) and 
fewer pther-eetarences, which is a 
pattern common in very young; children 
and indicative of social immaturity. 


Studying vocabulary. usage, in terms of 
common versus uncommon words, Lozar, 
Wepman & Hass (1972) found that the ` 
verbal output ,of retarded children 
was very much like that of no retard- 
ed children. However, they. also 
noted that there were considerable 
differences in the retarded mA 
usage of different parts of s eech. 
They collected speech samples| from , 
institutionalized retarded: “chiTaxen| 
and normal children matched on either 
mental age or chronological age by 
asking them to tell stories to TAT 
cards. The retarded children often 
spent more time describing the cards 
which led to a higher proportion of: 
nouns, articles, and quantifiers in 
their speech, On: the, other- hand, the 
normal children were more likely ‘to 


tell stories about the pictures, pro- . 


ducing more verbs, prepositicns, and 
conjunctions. i 


Previous researcn with nonretarded 
adults has suggested that the earlier 


. a word is estimated to have been ac- 


_ Adividual, the faster 
ral from memory 


quired by a 
the word is tv ‘<: 


(Carroll & White, ty /a, 1973b; Lach- 
“man, 1973; Lachman, Schaffer, & Henn- 


rikus, 1974y. cthereTore Winters and 


TH, 


¥ v 


5 Retarded child- 


Brzoska (1975) have suggested that 
the sequence in which words origin- 
ally entered semantic memory can 

be estimated from their relative 
speech retrieval. Using this proce- 
dure, they found that retarded per- 
sons have smaller lexical stores 
than would be expected compared to 
nonretarded persons of a similar 
mental age, but that they displayed 
a similar sequence of acquisition of 
lexical items. An unexpected finding 
however, was that the, degree of lag ` 
varied according to the class of 

the word, such that retarded per- 
sons were most deficient in words 
normally acquired earlier in the 
development sequence. This finding, 
though paradoxical, does support 
earlier findings that as mental age | 
increases retarded persons show more 
diversity in their lexical usage 
(Bartel, Bryen, Keehn, 1973). 


In a follow-up to this study, Winters 
and Cundari (1979) compared the 
speed of retrieval of words from the 
lexicon of institutionalized mental- 
ly retarded adolescents to the age- 
of-acquisition estimates derived 
from nonretarded adults. Words that 
were estimated to have been acquired 
the earliest by the normal group 
were retreived the fastest -by. the 
retarded group. The authors inter- 
preted these results as indicating 
that the verbal memory process of 
retarded and nonretarded persons 

are quite similar, despite the dif- 
ferences in lexical store size found 
by Winters and Brzoska (1975). 


Layton and Sharifi (1978) examined 
the basic semantic structure under- 
lying sentences produced by Down's 
Syndrome children and nonretarded 
children. Down's Syndrome children 
used the same semantic features as 
did the nonretarded children, but 
with much less frequency or consis | 
tency. For example, both groups 
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produced:-the four basic semantic verb 
categories (state, process, action, 
and process-action), bub the retarded 
group used the process (The road ends) 
and the process-action(John cut the 
paper) verb forms less frequently 

than the nonretarded chiidren did. In 


contrast, there was a tendency for the 


retarded children to use more state 
(The road is rough) and action (John 
runs) verbs than the normal group did. 
This finding was interpreted as indi- 
cating that’ Down's Syndrome children 
are more aware of or possibly restric- 
ted to the "here-and-now" aspects of _ 
their environment: "As children's cog- 
nitive abilities mature, their ability 
to perceive past and future events im- 
prove; consequently, their expressions 
will include more semantic concepts 
depicting these changes" (p. 444). 
Overall, Layton and Sharifi concluded 
that there are more similarities than 
differences in the usage of verbs by 
Down's Syndrome and normal children, 
and the primary cause of delayed se- 
mantic development among Down's children 
is their immature level of cognitive 
development. ; 


Recent research on how children use 


. words to convey meaning suggests that 


retarded and nonretarded children are 
more alike than different in this di- 
mension. Young normal children have 
been shown to use single-word utter- 
ances for more complex purposes than 
the simple labeling of an object. For 
example, Greenfield and Smith (1976) 
suggest that young children cali at- 
tention to a changing element or as- 
pect which is undergoing the great- 
est amount of change. Leonard, Cole, 
and Steckol (1979) reported that re- 
tarded preschool children label ob- 


jects in much the same way as the non- 


retarded children in Greenfield and 
Smith's study to convey the greatest 
amount of information efficiently. 


2.15 


Similarly, there is also some evidence 
to suggest that retarded children 

seem to impose meaning on verbal in- 
formation in much the same way as 
nonretarded children do. Nonretarded 
children frequently rely on context- ` 
ual information and their knowledge 

of real world relationships to inter- 
pret the meaning conveyed by a speak- 
er. Dewart (1979) designed a study tc 
determine whether mentally retarded | 
children used strategies similar to 
those used by nonretarded children 

for interpreting the meaning of sen- 
tences. Three types of sentences were 
used that varied acco.-ding to the 
kind of semantic cues they provided 
the child for use in interpretation: 
neutral, probable, or improbable. 
Sentences that were neutral with re- ` 
spect to semantic expectations would 
make sense if either noun served as 
the subject in the sentence. Probable 
sentences described likely events, 
while improbable sentences described 
events that were unlikely to occur in 
the real world (e.g., The gate jumped 
the horse). If children were using 
semantic constraints to interpret these 
sentences, they would respond correct- 
ly more often to probable sentences 
than to neutral sentences, and least 
often to inprobable sentences. Dewart 
concluded that retarded children are 
able to use contextual information 

and previous knowledge concerning the 
likelihood of a given event to guide 
their understanding of sentences in 
much the same way as nonretarded chil- 
dren ofa comparable mental age do. 


In summary, mentally retarded persons 
appear to demonstrate less mature iex- 
ical development than normal children do. 
They show less abstraction, fewer other- 
references, and greater attention to 
semantic features that are anchored in 
the present. They typically define words 
in more simplistic and concrete terms 
because abstraction develops much later 


‘in retarded populations due to cognitive 
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limitations. Some studies indicate 
that retarded persons have a smaller 
lexical store than normal. Even when 
they. are shown to be superior in vo- 
cabulary size, they typically demon- 
strate less variety and diversity in 
their lexicon. Despite these differ- 
ences, ‘the sequence of acquisition 
for lexical items has been shown to 
be the same as in normal development, 
only slower. The differences tend to 
diminish as mental age increases in 
the retarded population. 


In general, mentally retarded persons 
appear to be able to understand and 
use semantic information in much the 
same way as nonretarded persons of 
the same mental age do. They have 
been shown to use the same types of 
semantic features in their speech as 
nonretarded children, even though 
they use them less frequently and 
with less consistency. The retarded 
also tend to use similar strategies 
for interpreting meaning and convey- 
ing information to others. Therefore, 
the semantic system of mentally re- 
tarded children develops normally, al- 
though they tend to lag behind in 
their rate of acquisition. Retarded 
persons can generally be expected 

to perform much like younger normal 
children do, sometimes even more im- 
maturely than their mental ages 
would cause one to predict. Conse- 
quently, in developing interview 
questions for mentaily retarded con- 
sumers with .low mer.tal ages, one 
should, to the extent possible, use 
very simple sentence constructions 


which draw on concrete and commonly 


used words. 


LANGUAGE STRUCTURE OF THE MENTALLY 
RETARDED 

The structure of language used by 
mentally retarded individuals has re- 
ceived increasing attention in the 
literature over the past several], 


years. This aspect of language acqui- 


sition includes the areas of morphol- 
ogy, or rules for forming words, 

and syntax, or rules for forming 
grammatically correct and meaningful 


_ sentences. Recently, there has been 


increasing emphasis on the mentally 
handicapped person's ability to com- 
prehend language structures apart from 
his or her expressive abilities. In 
any attempt to obtain information 
from mentally retarded consumers 
about their needs, an understanding 
of their limitations in comprehending 
speech would be of great importance, 
since expressive requirements could 
be minimized by using various struc- 
tured interviewing strategies. 


Morphology 

Morphology is the study of word 
forms in a language. Berko's (1958) 
test of English morphology has be- 
come the most widely used method for 
studying the development of morphol- 
ogical rules in children, and has 
also been extended to investigating 
the provess of morphological rule- 
learning in the retarded. For exam- 
ple, the child is shown a picture of 
an imaginary object and is told "Here 
is a mook; now there is another one. 
There are two of them. There are two 

C" The child must provide the 
correct plural form of the nonsense 
word. 


In reviewing the studies which em- 
ployed Berko's technique with retard- 
ed children, Yoder and Miller (1972) 
and Cromer (1974) have concluded that 
retarded subjects matched on mental 
age learn the rules of morphology in 
a similar manner to normal children, 
but at a slower pace. However, some 
researchers contend that the retarded 
also differ from normals in their 
ability to generalize these rules to 
new, unfamiliar stimuli (Leiber & , 
Spitz, 1976: Lovell & Bradbury, 1967). 


A direct test of the rule-learning 


. deficit hypothesis was made by Bartel 
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(1970) with a group of mildly and 
moderately retarded children. She 
predicted that if such a deficit ex- 
isted in the retarded population, 
moderately retarded children would 
improve on specific words when train- 
ed to provide the correct inflections 
for them, but the improvement would 
not generalize to other words. Con- 
trary to her hypothesis, the two IQ 
groups in the study generalized to 
untaught items equally well. 


In light of evider.ce provided by 

Dever (1972) on the predictive val- 
idity of Berko's procedure with the 
mentally retarded, much of the pre- 
ceding information must be interpre- 


_ ted with caution. Consistent with 


previous findings, he and Gardner 
(1970) found that retarded children 
did not perform as well as normals 
on the Berko test even when matched 


“on mental age. However, they observ- 


‘ed that during spontaneous conver- 


sation some of their retarded subjects 


demonstrated appropriate use of many 
morphemes, even though they had per- 
formed poorly on the Berko task. 
Dever (1972), therefore, used a re- 
vised version of Berko's test to 
determine whether performance on ei- 
ther part of Berko's test could pre- 
dict the free speech errors cf re- 
tarded subjects. Since it could not, 
Dever concluded that the Berko test 
is not a valid diagnostic test of 
morphology usage in the mentally re- 
tarded. j 


However, it iş important. to note 

that retarded:and normal groups found 
the same items to be relatively dif- 
ficult in. Dever and Gardner's, study. 
Plurals and possessives were easiest, 
followed by progessive verb forms. 
Irregular verb forms were not hand- 
led well by any of the children be- 
fore mental age of 10. Therefore, 
despite the lack of predictive valid- 
ity of this test with respect to 


` morphological usage in spontaneous 


l 


speech, it appears that retarded 


-children develop understandings of 


morphology in the same sequence nor- 


mal children do. Overall then, while 


the: mentally retarded may sometimes 
demonstrate less flexibility in gen- 
eralizing morphological rules to new 
situations than normal children, 
there seems to be no strong support 
for the notion of a rule-learning 
deficit in retarded children. In 
addition, Graham and Graham (1971) 
reported that even severely retard- 
ed persons produce a considerable 
amount of speech which is formulated 
on the basis of appropriate grammat- 
ical rules and operate at a level 
comparable to that of normal chil- 
dren, of the same mental age. 


" Syntax 


In studies of syntax development 
in children one method that is fre- 
quently used is the cloze procedure, 
which requires the child to supply a 
word missing from a sentence. Perfor- 
mance on this task is determined by 
the child's ability to correctly 
judge the grammatical class of the 
word and then to select an approp- 


_xiate word from that class te com- 


plete the sentence. Research has 
demonstrated that retarded children 
are more likely to use sequential 
strategies rather than paradigmatic 
strategies in making word associa- 
tions (Semmel, Barritt, Bennet, & 
Perfecti, 1968). That is, like young 
normal children, they associate. 
words on the basis of contiguity (e. 
g., red-blue). For this reason, Sem- 
mel, Barritt, and Bennet (1970) pre- 
dicted that retarded subjects would 
find the cloze procedure more dif- 
ficult than the nonretarded children. 
They also predicted that when a word 
was omitted from the end of a sen- 
tence, retarded children would suc- 
ceed more often since more semantic 
clues would be provided to guide the 
selection of the missing word. 
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Retarded children ages 10 to 14 years 
with an IQ range of 60 to 80 were 
matched with two groups of nonretard- 
ed children: one on chronological age 
and one on mental age. In general, 
the retarded children did more poor- 
ly than either nonretarded group in 
the use of specific*grammatical form 
classes. AS predicted, the retarded 
children also showed greater improve- 
ment in performance when supplying 
words omitted from the end of a sen- 
tence than normal children, 
that the retarded children were more 
dependent on sequential strategies 
for selecting words. However, in re- 
viewing this study, Cromer (1974) ob- 
served that there was sufficient 
similarity in the performance of the 
retarded children and the normal 
children matched on mental aye to 
suggest that the retarded group was 
not responding with deviant strat- 
egies. For example, both the retard- 
ed and nonretarded children found 
nouns the easiest to replace. 


Goodstein (1970) replicated Semmel. 
et al. (1970) study with the addi- 
tion of a recognition version of the 
cloze procedure. These results in- 
dicated only quantitative differences 
between retarded and normal children. 
Both studies, then, suggest that 
sentence complexity affects sentence 


' + comprehension among the retarded in 
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much the same way as it does among 
normal children. Retarded children 


did not make different kinds of errors, 


just more of them. Consistent with 
the findings of Semmel et al. (1970) 


Goodstein found that the order of sen- 
tence difficulty was the same for both 
the retarded and nonretarded children. 


Several researchers have attempted to 
determine if the kinds of sentence 
constructions that normal children 


and adults have difficulty understand- 


ing also cause problems for the re- 


tarded. Semmel and Dolley (1971) 


studied a sample of Down's Syndrome 
children ranging in age from 6 to 14 


suggesting 
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years with an IQ range of 22 to 62. 
They presented their subjects with 
both comprehension and imitation 
tasks.using the four sentence types, 
and expected to replicate the order 
of sentence difficulty reported by 
Slobin (19665) for normal children 
and adults. In the comprehension 
task, subjects were presented with a 
pair of pictures, each representing 
one aspect of a reversible situa- 
tion (e..3., A clown hitting a ball . 
and a ball hitting a clown). As pre-- 
dicted, Down's Syndrome children 
performed best on the simpie de- 
clarative sentences (slightly above 
chance). Performance on passive and 
negative passive sentences was only 
at a chance level of responding. 
Surprisingly, Down's Syndrome chil- 
dren performed below chance on nega- 
tive sentences.. The authors suggest- 
ed that these children were respond- 
ing to negative. sentences as if they 
were simple affirmative sentences, 
and proposed that Down's Syndrome 
children may be unable to process 
negative ‘sentences, or may simply 
ignore the negative marker in these 
sentences. As a result, Slobin's 
findings were not replicated in this 
group of retarded children. 


In contrast, Lamberts and Weener 
(1976) reported that over half of ` 
the retarded subjects in a language- 
training project successfully repeated 
sentences which contained a negative 
marker. Since previous research has 
demonstrated that retarded children 
correctly imitate only those sen- 
tence structures which already exist 
in their own spontaneous speech pat- 
terns or which they are able to un- 
derstand (Lackner, 1976;,. Lenneberg, 
1967), the hypothesis that retarded 


‘children cannot process negative 


sentences seems less plausible. Moreover, 
in the Lamberts and Weener study, compre- 
hension of negative and affirmative sen- 
tences was related to sentence revers- 
ibility (that is, to whether subject and 
object can be interchanged, as in "the 
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cat chased the dog," or whether they 
are irreversible as in "the cat climb- 
ed the tree"). Nonrevėrsible nega- 
tive sentences were actually easier 
to understand than reversible affir- 
mative sentences for retarded per- 
sons with a mental age of five or 
above. 


Semmel and Dolley (1971) did not 

find sentence comprehension to be 
related to level of intelligence. 
However, later research has reported 
that the ability to comprehend sen- 
tences in retarded children does vary 
with Mental age (Bartel, Bryen, & 
Keehan, 1973; Lamberts & Weener, 1976; 
Walker, Roodin, & Lamb, 1957). Lamb- 
erts and Weener note that, on the 
average, the subjects in Semmel and 
Dolley's sample were much younger 
than their sample of retarded chil- 
dren. Therefore, comprehension of more 
complex sentence constructions may 
not have developed in Semmel and 
Dolley's Down's Syndrome children 
because a requisite level of cogni- 
tive or linguistic functioning had 
not been attained. Semmel and Dolley 
(1971) did find level of -intelli- 
gence among Down's children to be 
strongly associated with performance 
on a sentence imitation task. 


Mittler (1970) proposed that sen- 
tence comprehension depends more on 
structural complexity than on sent- 
ence length. He reported that the 
rank order of sentence difficulty 

was similar in a group of retarded 
children and a matched sample of 
nonretarded children. An analysis 

of errors suggested that comprehen- 
sion was most affected by the struc- 
tucal complexity of the sentence in 
both groups of children. Berry (1972) 
examined the effects of structural 
complexity on sentence comprehension 
when sentence length was controlled. 
He compared the comprehension ability 
for retarded and nonretarded children 
for simple declaratives (e.g., The 
boy is fishing here.) and more com- 


_ plex possessive sentence construc- 
- tions (e.g., The boy’s fish is here.) 


Possessive sentences were more 
difficult for both groups of chil- 
dren. 


A number of other studies have shown 
that retarded children are generally 
able to process simple declarative 
sentences with the same accuracy as 
nonretarded children of a comparable 
mental age. However, as transforma- 
tional complexity increases, the 
retarded child lags further behind 
his or her mental age counterpart in 
sentence comprehension. For example, 
Dewart (1979) reported that retarded 
chilé@ren above mental age three com . 
preheud simple active sentences’ above 
chance, while passives are inter- 
preted at a level below chance. She 
indicated that their, performance 
was very similar to that of a group 
of nonretarded children whose av- 
erage mental age was lower, which 
supports the idea of delayed acqui- 
sition of passive constructions in 
the retarded. Retarded children 
with a lower mental age, however, 
performed at a chance level on both 
active and passive sentences. 
Therefore, Dewart concluded that, 
prior to mental age three, retarded 


‘children are quite limited in their 


ability to understand and use 
syntax. 


Despite the fact that the acquisi- 
tion of more complex sentence struc~ 
tures is delayed in the language 
development of the mentally retarded, 
such learning does occur. The order 


' of acquisition of complex sentence 


constructions parallels that ob- 
served in the developmental sequence 
of normal children ‘(Berry & Foxen, 
1975; Gallagher, 1969). For example, 
Graham and Graham (1971) collected 
language samples of nine institution- 
alized retarded males, ages 10 to 18 
years, with mental ages ranging from ' 
3-6 to 10-0. Retarded males with 
lower mental ages produced most of 
their sentences through the use of 


Q 
ERIC 


base strings or kernel sentences 
with no transformational rules ap- 
plied. Mental age was highly corre- 
lated with use of transformations to 
form more complex sentences. Thus 


Graham and Graham concluded that 


language facility does depend pri- 
marily on intellectual level in 
retarded persons and that retarded 
children develop the rules of lan- 
guage in much the same manner as 
nonretarded children only at a slow- 
er rate. l 


In one of the most linguistically 
sophisticated studies to date on the 
language development of retarded 
children, Lackner (1976) has provid- 
ed additional support for this view- 
point. He collected language samples 
from five retarded children who 
represented five ascending levels of 
intellectual functioning and com- 


, pared their linguistic capabilities 


with those of nonretarded children. 
The order of appearance of sentences 
in the language patterns of the re- 
tarded children was consistent across 
the five mental age levels and fol- 
lowed a regular order of increasing 
structural complexity: declaratives, 
negatives, questions, negative ques- 
tions, passives, negative passives, 
and negative passive questions. No 
sentence type of a given order of 
complexity was found in the retard- 
ed child's grammar unless all lower 
order constructions were also part 
of his or her language repertoire. 
More significant was the finding 
that as mental age increased the 
number of. sentence transformation 
roles understood and used by each 
child also increased, indicating that 
language development as measured by 
transformational complexity does 
improve with higher intellectual 
functioning. This Supports the pre- 
vious findings of Graham and Graham 
(1971). Lackner also observed that 
none of the sentences of the retard- 


ed children were incompatible with 


`- normal adult usage, supporting the. 


idea that retarded persons do not use 
deviant language structures. Retarded 
and normal children exhibited simi- 
lar developmental trends, but the 
language development of the most 
severely retarded children appeared 
to be arrested at a level lower than 
that of their mental age-matched 
counterparts. 


There has been much debate in the 
literature over the relationship be- 
tween comprehension and production 

in the language acquisition process. 
Current thinking assumes that compre- 
hension precedes expression or pro- 
duction in the developmental sequence. | 
Research on language development of 


‘nonretarded children, supports this 
‘notion (Fraser, Bellugi, & Brown, 


1963; McCarthy, 1954). Other recent 
studies with nonretarded children 
have also demonstrated that prior 
training in comprehension skills fac- 
sllitates acquisition of production 
abilities (Vasta & Teitelbaum, 1976; 
Whitehurst, 1977).- 


In contrast, some studies with men- 
tally retarded persons have reported. 


_different and conflicting informa- 


tion. For example,Guess (1969) and 


Guess and Baer (1973) concluded that lan- 


guage reception and production in 
mentally retarded children may be 
independent processes. They found 
that training retarded subjects in 
one modality did not produce general- 
ization effects in the other language 
modality. Similarly, Miller, Cuvo, 
and Borakove (1977) found that teach- 
ing reception skills to the retard- 
ed did not improve expression; how- 
ev2r, training in production did im- 
prove receptive skills. This research 
suggests that a qualitative differ- 
ence may exist between retarded and 
nonretarded populations with respect 
to the direction of generalization be- 
tween comprehension and production. 
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Lackner (1976), however, feels that 
production of grammar lags behind 
comprehension in the mentally re- 
tarded. He observed that both re- 
tarded and nonretarded children in 
his study frequently modified com- 
plex sentences on an imitation task 
in such a way that the meaning of 
the sentence was retained but sen- 
tence complexity was reduced. For 
example, a child would sometimes re- 
peat the passive sentence construc- 
tion "Mary was hit by the ball" 

as "The ball hit Mary." Lackner 
felt that this indicated that the 


‘child's productive skills for gram- 


mar were not as well developed as 
his ability to understand syntax. 


Similarly, Walker, Roodin and Lamb 
(1975) reported that expressive 
language consistently lagged behind 
comprehension in a sample of re- 
tarded children with a mental age of 
3 to 7.5. In a group of nonretarded 
children with a comparable mental 
age range, this difference occurred 
for only the youngest children, 

ages 3 to 3.5. As mental age in- 
creased in the retarded group, the 
magnitude cf the difference was 
found to decrease so that by mental 
age seven expressive and receptive 
abilities were nearly equal. The 
authors concluded that expressive 
tasks may place a greater demand on 
short-term memory than comprehen- 
sion tasks do. In their ‘research, 
"maximal estimates of language 
ability in retarded subjects were 
obtained in tests that measure 
comprehension without requiring 
verbal production by the child" 
(p.551). Consequently, one might 
expect that maximal information 
could be obtained from retarded per- 
sons using interview questions 
which make minimal demands for 
verbal production. 


In summary, then, the research evi- 
dence suggests that the language 


- rect grammatical usage. 


structures of mentally retarded 


persons. are quite similar to 

those of nonretarded children at a 
similar mental age. In addition, 
there is no strong evidence to 
suggest that the mentally retarded 
use language structures or strat- 
egies that are incompatible with 
those of normal children, even 
though language acquisition fre- 
quently occurs at a-much slower 
rate. Language delays are likely 
to be greatest for retarded persons 
below mental age seven, since 

prior to this level of functioning 
they do not seem to possess the 
cognitive capabilities necessary 
for comprehending the more complex 
aspects of grammar. In addition, 
retarded persons below mental age 
three would be expected to have 
very little understanding of cor- 
Since 
retarded persons have been shown 

to consistently perform best on 
language tasks involving simple 
sentence structure, this would 
suggest that unnecessary complexity 
should be avoided when questions 
are asked of the retarded. 


Question Development in Children 
and Its Impiications for the 
Retarded 

No research that we are aware 
of has directly examined the devel- 
opment of question-answering com- . 
petence in retarded individuals. 
However, research on normal chil-; 
dren's comprehension and. use of 
questions has been reported within 
the last 15 years, and it is highly ' 
relevant to our concern with how 
to phrase questions. The evidence 
suggests that there is a basic 
sequence of question development 
in normal children, although indi- 
vidual studies have shown some 
variation in the’order of mastery. 








It seems’ reasonable to expect that 
a similar pattern of questicn de- 
velopment occurs in the mentally 
retarded, Since their language pro- 
“cesses have not been shown to be 
. qualitatively different from those 
of nonretarded children. 


By age four, children are typically 
able to respond appropriately to 
most question forms. Bellugi (1965) 
observed that before age two cnildren 
do not seem to understand many of 
the wh-questions. Comprehension 
for these question formats begins 
to emerge somewhere around 18 to 

28 months of. age, with yes-no, ` 
what- and where-questions generally 
being the first to be understood. 
Why- and who-questions were among 
the next questions to be acquired 
(around the age of 32 months). 

These observations correspond very 
closely with =n early study by 

Smith (1933), who found that what, 
where, and yes-no question types 
were the most frequently produced 
by children ages 1-6 to 6-0. 


Ervin-Tripo (197C) conducted the 
first wajor study cf how children 
comm.chend questions. She observed 
tne responses tc. questions of chil- 
Aren ages 2.5 to 3.1 over 2 one-year 
period, Initially, all children 
couid answer what, where, and yes- 
no questions appropriateiy, which 
agrees with Bellugi's observations. 
-Ihe relative order of acquisition ~ 
for other wh-questions, according 
to Ervin-Tripp was: why and who- 
‘subject; how and where-from; when 
and who-subjevt questions. How- 
ever, Since this order was not 
stable across all children in the 
study, she presents this as a ten- 
tative developmental sequence for 
question acquisition in normal 
children. 


The level of the child's cognitive 
development appears to influence 


elaboration. 


what questions he or she can answer. 


For example, around the age of three, 
most children begin to produce ad- 
equate causal responses. to simple 
why questions that are manageable 

at their level of cognitive func- 
tioning (Bellugi, 1965; Ervin- 

Tripp, 1970). By comparison, the 
child's understanding of time 
concepts does not develop fully 
until after age six (Piaget, 1927), 
which would explain why when-questions 
are among the most difficult for 
young children to understand. 


Recently, Tyack and Ingram (1977) 
attempted to replicate the findings 
of Ervin-Tripp in a study of chil- 
dren's question comprehension and 
production. The children in the 
production experiment used yes-no, ` 
what, and where questions most 
frequently, with the exception of 
3-6 to 3-11 group who used why and 
how questions more frequently than 
where questions. The authors 
suggested that what and where ques- 
tions are used frequently by young 
children because they are closely 
tied to the child's immediate ex- 
perience. An alternate explanation 
has been suggested by Hooper (1971), 
who proposes that yes~no and label- 
ing questions like "What is that?" 
are much easier for children to 
answer than open-ended and explana- 
tory questions such as how and why 
because they make fewer demands , 
for grammatical processing and verbal 
Possibly both this 
linguistic explanation and the cog- 
nitive development viewpoint account 
for the earlier use and comprehen- 
sion of yes-no and what questions 

in children. - l 


While frequency of use does nọt 
necessarily reveal the order of 
question acquisition in children, 
the usage figures reported by Tyack 
and Ingram (1977) do agree with the 
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comprehension data of Bellugi (1965) 
and ‘Ervin-Tripp (1970). However, 
the acquisition sequence for ques- 
tion comprehension which emerged 
from Tyack and Ingram's sample of 
3.0 to 5.5 year olds was somewhat 
different from that reported in 
earlier studies. The order of ques- 
tion type was as follows: where- 
intransitive (94%); why-intransitive 
(83%); why-transitive (81%); 
who-subject (80%); where-transitive 
(67%); «shat-object (57%), who-object 
(56%); when-intransitive (52%); 

and when-transitive (48%); how- | 
transitive (38%); how-intransitive 
(35%), and what-subject (35%). 
According to these findings, intran- 
sitive verb forms are better under- 
stood than transitive forms, except 
in how-questions. As the authors 
note, when-questions were easier to 
answer than how-questions for these 
subjects, where this was not the 
case in Ervin-Tripp's study. A 
more obvious discrepancy was that 
what-questions were more difficult 
to answer in this study. In fact, 
what-subject questions were actually 
the, most difficult to understand 
for these children. The authors 
suggest that young children may 
automatically associate what 
questions with the object. More 
generally, Tyack and Ingram con- 
cluded that transitivity and the 
semantic features of a verb are as 
important in determining the child's 
response to a question as is the 
question format used. 


The research evidence on the acqui- 
sition and comprehension of questions 
in normal children ‘seems to suggest 


that the mentally retarded, especially 
‘those who are severely retarded, 


would find what, where, and yes-no 
questions the easiest to comprehend 
-, They would probably 
find who, how, and why questions 
more difficult to comprehend and 


even more difficult to answer be- 
cause Of the greater demands for 
verbal expression made by these 
question formats. When-questions 
would probably be the most difficult 
to answer because of the higher 
level of cognitive functioning nec- 
essary to understand time con- 
cepts. However, as Tyack and Ingram 
have indicated, the transitivity 

and semantic features of the verb . 
used could also affect this order 
of difficulty significantly. 

Since level of cognitive develop- 
ment appears to be an important 
factor in the normal child's 

ability to respond to questions, 

one would expect retarded children 
to acquire these question forms 

at a slower rate, especially if they 
are severely retarded. ` 


Response Effects in Survey Research 
In any research, including re- 
search using interview techniques, 
validity of measurement- is a 
critical issue. Simply stated, 
one needs to know what an inter- `> 
view measures; that is, whether re- 
sponses to an inťèrview accurately 
portray the actual behaviors or 
attitudes of the individual inter- 
viewed. Since part of the purpose 
of our research was assessment of 
the validity of information ob- _ 
tained from mentally retarded re- 
spondents, we sought guidance from 
the literature on measurement and 


`~ measurement error in survey research. 


Traditionally it has been assumed 


that responses to an interview are 


composed of two factors. The first 
is the respondent's "true answer," 
or the actual information sought by 
the interviewer. The second is 
response error, or variability 

in the respondent's answer due to 
all sources other than the "true 
answer." Response error, then, is 
the difference between the "true 


answer" and the answer actually 
given in the interview. Response 
error arises from many sources; 

a response effect is the response 
error due to one specific factor. 
If response effects tend to cancel 
one another out so that mean re- 
sponse error is zero, straight- 
forward interpretation of the: 
interview data for a group is pos- 
sible. If, however, some response 
effects are large enough that the ` 
obtained answer consistently” 
varies in a given direction from 
the "true answer," interpretation 
of the interview data is much more 
difficult. Data from our research 
suggest that such powerful response 
effects may be at work in inter- 
views with -mentally retarded indi- 
viduals, tending to reduce the 
validity of answers. The litera- 
ture review which follows is an ef- 
fort to assess the relative impor- 
tance of response effects due to 
various factors, with particular 
emphasis on those variables ex- 
pected to have a special importance 
in interviews with mentally re- 
tarded persons. We will be con- 
cerned with learning whatever les- 
sons may be learned.from survey 
researcn with the general popula- 
tion that might aid in interview- 
ing mentally retarded respondents. . 


Fortunately, Sudman and. Bradburn : 
(1974) have provided a review and 
analysis of research of response 
effects which is a landmark in the 
field. These authors used a-coding 
strategy to combine results of an 
enormous number of interview 
studies and estimate the relative 
importance of various response 
effects. The present review is, 

in many respects, a summary of the 
major conclusions of the Sudman and 
- Bradburn work, with emphasis and 
elaboration of those biases that 
seem potentially most relevant to, 


, 


interviewing mentally retarded 
persons. In particular, the model 
of the interview situation and 
the conceptual framework for 
evaluating different types of 


‘response effects are adopted 


directly from Sudman and Bradburn. 
This model views the interview 

as composed of three elements: 

the task of giving and receiving 
information, the role of the inter- 
viewer, and the role of respondents. 
Thus, the model regards response 
effects as a function of either 
task variables, interviewer vari- 


ables, or respondent variables. 


'Task'variables are the form and 


conditions of the interview itself. 
The specific characteristics of 

the task may be expected to exert 

a tremendous influence on the 
nature of the information obtained. 
Question wording is an important. 
task variable. Bias may be intre- 
duced directly by. the wording of 

a question, Or may result indi- 
rectly from the length or diffi- ` 
culty of questions. One might also 
expect the structure of questions 
(e.g., closed-ended vs. upen-ended) 
to exert an effect on responding. 
Questions which offer the possibility 


of an agreeing response can give 


rise to powerful acquiescence 


_effects, a tendency tO agree or 


say "yes" regardless of question 
content. Another task variable is 
the degree to which an interview 
raises issues Of self-presentation 
or, more specifically, creates 

in respondents the desire to 
present themselves in a favorable 
light. Whether an interview is 
directed towards behavioral or 
attitudinal information is another 
task variable of interest.. Still 
another may be described as the in- 
formational demands of the interview. 
Questions may place greater or 
lesser demands on the’respondent's | 
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memory, and may pertain to topics 

of greater or lesser salience to the 
respondents. Finally, the method of 
administration (for example, face- 
to-face vs. self-administered inter- 
views) is potentially important. 


Interviewer and respondent variables 
generally refer to characteristics 
of the individuals involved, such as 
their age, race, sex, and level of 
education. The magnitude of response 
effects due t^ interviewer character- 
istics is usually dependent on 
respondent characteristics and the 
topic of the interview. Similarly, 
the importance of respondent var- 
lables typically depends on inter- 
actions with task and interviewer 
variables, In fact, although for 
the sake of clarity Sudman and Brad- 
burn describe task, interviewer, and 
respondent variables as separate 
sources of response error, their 
data show clearly that most impor- 
tant response error is a function of 
combinations of these variables. For 
example, while question wo Jing and 
the education of the respondent 

can each give rise to response ef- 
fects, the response effect due to 
the interaction of these two var- 
lables is larger. 


TASK VARIABLES ; 
The trend in survey research 
has long been to neglect task vari- 

ables in favor of investigations 

of interviewer and respondent 
characteristics. The most general 
conclusion of the Sudman and 
Bradburn review is that this em- 
phasis is unwarranted; in fact, the 
most general and important resp:nse 
effects arise from task variables. 


The specific task variables found to , 


be of importance are evaluated below. 
Much of our own research effort fo- 
cused on the effects of task vari- 
ables such aS question structure and 
wording. 


Ñ 


Question wording 

The specific wording of a ques- 
tion might be expected to give rise 
to response effects in two general 
ways. First, bias inherent in the 
wording, as in the use of emotion- 
ally laden or leading words, may 
alter the respondent's answer. Sec- 
ond, difficult or ambiguous ques- 
tions may enhance response effects 
due to other variables. Essential- 
ly, if the respondent cannot inter- 
pret the question, he or she cannot 
give the "true answer. ' 


Good evidence exists that responses 
can be systematically biased by 
question wording. For instance, 
Rugg (1941) asked the questions: 
"Do you think the United States 
should allow public speeches — 
against democracy?" and "Do you 
think the United States should for- 
bid public speeches against democ- 
racy?" In response to the first 
question, 62% of respondents be- 
lieved such speeches should not be 
allowed, while in answering the 
second, only 46% were willing to , 
forbid speeches. Clearly, the word- 
ing of these questions had a strong 
influence on the response effects 
due to question’ wording. 


Schuman and Presser (1977) repli- 
cated and extended these results. 
Using Rugg's original questions, 
they found response effects due to 
questions wording to be of similar 
magnitude and direction. Addition- 
ally Schuman and Presser challenged- 
the assumption, implicit in much re- 
sponse error literature, that ques- 


‘tion wording and respondent charac- 


teristics do not interact. If this 
assumption were correct, ‘then re- 
sponse effects due to question word-. 
ing would not affect comparisons , 
of subgroups of respondents. How- 
ever, Schuman and Presser showed 


that less educated respondents were 
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“more influenced by question wording 


than were more educated respondents. 


' Among respondents with 13 or more 


years of education, the "allow" and 
"forbid" forms of the question 
yielded estimates of the proportion ` 
of respondents opposing speeches 
which differed by only 5.9%. This 
difference was 25.6% for respondents 
with 0-11 years of education. In 
short, question wording can iave 
powerful response effects, and these 
response effects are greater for less 
educated respondents. These results 
may have important implications for 
the interviews with mentally retarded 
individuals, who are at the low end 
of the educational continuum. 


The length and difficulty of ques- 
tions also may affect responding. 
Length of question, Simply defined 

as the number of words, has generally 
had no overall effect on responding 
in face-to-face administration. Of 


.more interest for our research is the 


interaction of question length with 
respondent variables. Respe.ise ef- 
fects for respondents with a high 
school education or less are largest 
for questions more than 18 words 
long. Elementary students are par- 
ticularly influenced by long ques- 
tions. This finding may have impor- 
tant implications for interviewing 
mentally retarded persons; if long 
questions cause large response ef- 
fects in younger and less educated 
respondents, it may be particularly 
important that questions posed of 
mentally retarded individuals be 
short and clear. , : 


The number of letters per word has 
been used as an indicator of question 
difficulty. Difficulty of questions 
has no,general-effect for attitudinal 
items; although response effects for 
threatening behavorial ittems increase 
for questions using longer words. 


Question Structure 

The issue of question structure 
is typically posed as a choice þe- 
tween closed-ended questions, which 
ask the respondent to choose from 
a fixed set of response alternatives, 
and open-ended questions. Although 
Sudman and B-adburn found no consist- 
ent effect due to question struc- 
ture, they did find that response 
effects due to issues of self- 
presentation are larger for closed- 
ended than for open-ended questions. 
That is, for threatening topics or 
topics for which there is a socially 
desirable answer, closed-ended ques- 
tions increasd response effects. 


In addition, direct comparisons of 
closed and open-ended surveys have 
demonstrated that different 

forms of a survey can yield very 
different results. Jenkins (1935) 
reported a study which involved zon- 
struction of an exhaustive checklist 
by using all responses to a previ- 
ously administered open-ended survey. 
This checklist (including an "all 
others" items) yielded response 
patterns closely comparable to the 
open-ended survey. However, when 
some of the popular items’ were 
deleted from the checklist, response 
patterns to the incomplete check- 
list form differed radically from 
responses to the open-ended form. 

It is.at least clear that the make- 
up-of a checklist can have important 
influences on responding, and that 
if items often mentioned in response 
to open-ended questions are omitted, 
under-reporting may result. 


` Belson and Duncan (1962) compared 


checklist and open-ended surveys 
with regard to TV programs watched 
and periodicals read by respondents 
during a specified period preceding 
the interview. They reported far 


higher claims of activity in response 
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to the checklist form than to the 
open-ended survey, with the size. of 
differences between question forms 
varying across items. Also, Belson 
and Duncan included some checklist 
items naming TV programs not actu- 
ally televised during the period 

of inquiry. A small fraction of 
respondents claimed to have watched 
these programs, demonstrating that 
in at least some instances closed- 
ended surveys can cause overreport-— 
ing. 


Schuman and Presser (1977) also 
reported large differences on some 
items between endorsement rates on 
open and closed forms. Response 
effects due to question structure 
were found to be related to the 
education of the respondent. For 
example, the poorly educated were 
more likely than other respondents 
to endorse a job security item on 
the closed-ended form, but 

less likely to list the same item 
on the open-ended form. Response 
effects were also larger for the 
less educated in a comparison of 
agree-disagree vs. forced choice 
questions. 


In summary, although Sudman and 
Bradburn observed no general re- 
sponse effect due to question struc- 
ture, direct comparisons reveal sub- 
stantial response differences be- 
tween closed and open-ended ques- 
tions. Closed-ended (checklist) 
questions tend to provide a higher 
yield of clains, which in some _ 
instances may constitute overreport- 
ing.. Finally, response effects due 
to question structure may be exag- 
ge.cated in less educated responä- 
ents, another hint that special 
problems may arise in interviewing 
retarded persons. 


‘Acquiescence: 


Questions which allow for the 
possibility of an agreement response 


and rather complex. 


"yes," "true," "agree," and the like, 
may give rise to acquiescence, or a 
general tendency to agree independ- 
ently of item content. Since 
acquiescence was examined in the 
present study, we WAI consider it 
in some detail. 


unlike most response effects dis- 
cussed thus far, acquiescence is a 
topic of interest primarily within 
personality research. The large body 
of literature on ‘acquiescence (see 
Cronbach, 1946, for his influential 
introduction of the issue) emphasizes 
the effects of acquiescence on the 
interpretation of personality inven- 
tories like the California F scale 

and the MMPI. For example, items on 
the F scale are all keved true; it 

has been suggested (see, for example, 
Bass, 1955) that the F scale identi- _ 
fied particularly acquiescent indi- 
viduals, rather than authoritarian 
individuals as argued by the creators. 
of the scale. However, this body. Of: 
literature arrives at no consensual he 
conclusion regarding the importance | 
of acquiescence in personality inven- 
tories (see Rorer, 1965, fora eriti-; 
cal review of the literature). 


No consensus has been reached on this `. 
issue largely because it is a meth- 
odologically difficult one. In order 
to evaluate the importance of acqui- 
escence in any instrument, one must, 
somehow separate question form and | 
question centent--ordinarily highly 
confounded factors. This separation 
is particularly difficult with per- - 
sonality inventories, where the "true" 
answer to an item is now known and | 
where items are usually attitudinal 

It is therefore 
not particularly suprising that the. 
role of ‘acquiescence in these inven- 
tories is- unclear. 


Our research is directly concerned 

with acquiescence | as. a: factor: which 
might invalidate responses ‘to ques- 
tions much less ` poems (pou than those 
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typically found on personality in- 
ventories. The questions we are 
‘concerned with tend to be behavioral 
(Do you date?) or relatively 

simple attitudinal (Are you usually 
happy?) questions. As we will dem- 
` onstrate later, acquiescence may 

be a highly significant biasing 
factor in interviewing mentally 
retarded persons. The research de- 
scribed below is organized around 
four approaches to assessing acqui- 
escence. 


Content-free Measures 

One strategy for demonstrat- 
ing the effect of acquiescence is 
to employ "interviews" which are’ 
essentially content-free. In this 
.Situation, any non-random response 
patterns observed must be responses 
to the form rather than the content 
of the question. For example, 
Berg and Rapaport (1954) gave sub- 
jects a set of response alterna- 
tives without any questions. They 
were instructed to try to guess 
the correct answers. Berg and 
Rapaport observed clearly ncen- 
random patterns of responding to 
this essentially content-free 
measure. Of particular interest 
is the fact that response alterna- 
‘tives "Yes," "satisfied," "true," 
and "agree" were significantly 
favored by respondents regardless of 
the order in which alternatives were 
presented. 


In a similar study , Gerjouy and 
Winters (1966) tested response 
preferences in 60 institutionalized 
educably retarded adults. They 
presented the subjects with pairs 
of identical geometrical figures 
and asked if the figure to which 
they pointed was the larger (or 
smaller). Of all responses, 59.1% 
were "yes," significantly different 
from chance at the .001 level. 


Clearly, a preference for agree- 
ment responses exists in both nor- 
mal and retarded individuals in 

an extremely unstructured situ- 
ation. It is conceivable that 
questionnaires may essentially 

be unstructured or "content-free" 
for mentally retarded respondents 
if the demands of the. questions 
exceed intellectual abilities; in 


.such a case, responding would 


probably tend to resemble the 
acquiescence observed in. the above 
studies. 


Tests with Known Answers 

A -second strategy, one employed 
in the present study, is to use 
items to which the correct answers 
are known. If iñ this situation 
the pattern of responding differs 
systematically from the pattern of 
correct answers, a response set 
may be inferred. Cronbach °(1942) 
used this approach with true-false 
tests and observed that students 
in general are inclined to guess 
"true" when in doubt. Further, .- 
Cronbach found that each student's 
tendency to acquiesce was somewhat 
consistent across tests; a given 
student's tendency to answer "true" 
correlated from .36 to .61 across 
tests. Interestingly, if students 
guess "true" when in doubt, then 
false items have greater validity 
than true items as measures of the 
students’ knowledge. In fact, 
Cronbach showed that in some cases 
the false items have greater 


validity than the true and false 


items combined. This finding may 
be important in survey research 
with particularly acquiescent 
populations; from such persons, 

a "no" response would, on the 
average, be more meaningful than 

a "yes" response. In short, in- 
dividuals have a somewhat: stable 
tendency to answer "true" when in“ 
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doubt, and this fact tends to common flaw in item-reversal re- 
make false items more valid than search), but did report that con- 
true items. tradictions were "mostly" of the 
agree-agree type. This result in- 
The Item-reversal Approach dicated that a tendency on the part 
Unfortunately, neither of the oe certain types of respondents 
; to acquiesce can generally in- 
above research strategies can be f v : ; 
applied in most practical con- validate: antenvrew Gare 


texts. In interview situations, One study directly investigated 
items have content and the correct the effects of acquiescence with 
answer is almost always unknown. mentally retarded respondents. 
The item-reversal appr ach has Gozali and Bialer (1968), working. 
been devised for use in such cir- with a personality inventory called 
cumstances.. If an item can be Children's Locus of Control Scale, 
administered in two forms so that reversed the 23 items on this 
the content is precisely reversed scale and administered both forms ` 
in the two administrations, self- to a sample of 189 mentally retarded 
contradictions in which the re- individuals of both sexes. Sub- 
spondent agrees with both state- jects ranged in ace from 16 to -30 
ments are a clear indication of years and in IQ from 58 to 91. 
acquiescence. For example, if a The measure of locus of control 
“respondent replies "yes" to both derived from the standard form of 
"I earn more than $5000 per year," this test was highly correlated 
and "I earn $5000 or less per year," with the measure of locus of control 
` response is clearly based on ques- derived from the reversed form; 
tion form rather than on question that is, responding to the scale 
content. This technique gives. appeared to be based on item con- 
the most definitive demonstra- tent rather than item form. Un- 
tions of acquiescence, and was © - ~~ fortunately, Gozali and Bialer ' 
employed in the present study. -~ reported only correlation coeffi- 
Its -only major difficulty is that cients, mixed questions keyed "yes" 
clear content reversal of many and "no" on both forms, and did 
items, especially complex atti- not report self-contradictions on 
tudinal items, is virtually im- an item-by-item basis, leaving us 
possible. Rorer's (1965) critique uncertain about the extent to 
of item reversals with the F scale which acquiescence might have been 
makes this point quite convincingly. operating. 

Hare (1960) administered original Lenski and Leggett (1960) imbedded 
- and reversed forms of a 44-item a single question and its reversal. 
interview with an "agree"-"dis- ("It's hardly fair to bring chil- 
agree"-"can't answer" response dren into the world, the way things 
format. Subjects were black and look for the future," vs. "Children 
white wome. Self-contradiction born today have a wonderful future 
was observed, particularly among to look forward to,") in a long 

black women, who gave contradictory interview, separated by about 

answers to an average of 20 of one-half hour. For the sample as 

the 44 pairs of items. Hare did a whole, 8% of respondents agreed 

not separately report rates for with both items. This tendency 

contradictions of the agree-agree to acquiesce was related to both 

and disagree-disagree types (a education and race. Overal, 20% 
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of black versus 5% of white re- 
spondents agreed with both state~ 
ments. For white respondents, 
only 2% of those with some college 
education acquiesced, while 9% 

of those with 8th grade education 
or less acquiesced. Among black 
respondents, 14% of those with 
some college agreed with both 
items, compared to 32% of those 
with 8th grade education or less. 
Thus, black respondents and less-. 
educated respondents showed a more 


.marked tendency to acquiesce. 


Rothenberg (1969) used item- 
reversal in studies of cognitive 
development in young children. 
Studying conservation of number, 
Rothenberg asked his subjects 
both "Does this bunch have the same 
number of blocks as this bunch?" 
and "Does this bunch have more 
blocks?" Subjects were 4- and 
5-year-olds, divided into lower 
and middle class (distinguished 
by neighborhood). This class 
distinction also divided the 
sample according to race, for the 
lower class group consisted of 
black and Puerto Rican children, 
while the middle class sample 
included no Puerto Ricans “and 
only 4% blacks. Additionally, 
the class disti‘ction appears to 
have divided the sample according. 
to intelligence. As measured by 
the Peabody Picture Vocabulary 
Test, the 4- and 5-year-old lower 
class children had average IQ's 
of 72.2 and 72.5, respectively. 


-The middle class 4- and 5-year-olds 


had IQ's of 110.6 and 101.9. On 


‘a single test of conservation, 


65.1% of lower class 4-year-olds 
contradicted themselves by answer- 
ing "yes" to both forms of the 


“question stated above. The ac- 


quiescence rate was 40.0% for lower 
class 5-year olds, 7% for middle 
class 4-year olds, and 10.0% for 
middle class 5-year olds. The dif- 
ference between average acquies- 


ment of such scales typically in- 
"volves constructing a group of 


cence for the lower middle class 
samples was significant at the .001 
level. Because race, social status, 


‘and IQ are confounded in this design, 


it is impossible to determine which 
of these variables was responsible 
for the difference ‘in acquiescence 
rates in the two groups. However; 
this is a clear demonstration 

of the potentially ruinous effects 
of acquiescence on validity. 

For the sample as a whole (N = 210), 
27.8% of all responses were yes~yes 
s:lf-contradictions. (This far 
ouv.veighs the incidence of no-no 
self-contradictions, at 5.2%.) 

The fact that this very high 
acquiescence rate occurred among 
young respondents and was especially 
pronounced for respondents of ‘low 
intelligence at least suggests 

that acquiescence may be a parti- 
cular problem in interviewing 
mentally retarded individuals. 


Measures of Individual Differences 
in Acquiescence 

A fourth approach to the. eval- 
uation of acquiescence involves 
the construction of scales spe- 
cifically intended to measure the 


acquiescence tendencies of dif- 


ferent individuals. The develop- 


items which is held to be hetero- 
geneous with respect to. content. 
Furthermore, items are balanced 

so that equal numbers of agreeing 
and disagreeing responses are 
requireé. for expression of an 
extreme preference within any 
content area. It is assumed that 
the influence of content on the 
total number of agreement responses 
is thus eliminated, and the agree- 
ment score is taken to be a measure 
of inclination to acquiesce. In 
essence, this is simply an imprecise 
form of the item-reversal strategy. 
Couch and Keniston (1960) created 
‘such an acquiescence scale, con- 
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sisting of 360 items drawn from 

a variety of personality instru- 
ments and using a Likert type 

scale. Couch and Keniston attempted 
to correlate acquiescence as a 
personality variable with a number 
of other variables, constructing 


- a. personality profile for the 


yeasaying respondent. For our 
purposes, their most significant 
finding was that yeasaying and 
intelligence are unrelated. How- 
ever, Couch and Keniston used 
crude measures of intelligence 
(the intelligent-dull scale of 
the 16 P.F., and the College En- 
trance Examination Board scores) 
and used a sample with a narrow 
range of intelligence (subjects 
were college undergraduates). 
Thus, it is not surprising that 
yeasaying and intelligence were 
unrelated in this study. 


Wells (1963) derived a 20-item 
scale purported to measure ac= 
quiescent tendencies based on the 
earlier scale constructed by Couch 
and Keniston (1960). Wells re- 
ported that yeasayérs, as defined 
by reponse to his scale, generally 
tend to endorse a wide variety of 
behavioral items. Further, Wells 
found that yeasayers tend to over; 
report in surveys investigating 
such topics as possession of maga~ _ 
zines and recall of advertisements. 
Finally, it is interesting to ote 
that Wells found yeasaying parti- 
cularly common among younger and 
less-educated respondents. 


It is tempting to interpret this 
latter finding as lending strong 
support to the findings of the item- - 
reversal literature. However, 

this individual differences research 
must be’ interpreted very carefully; 
the imprecision of the content 
balancing procedurés opens these 
scales to other interpretations. 
Inspection of the items of the Wells 


mn eae 


(1963) scale suggests that rather 
than being "content balanced," 

this scale may in fact test for 

an attitude of optimism or uncritical 
enthusiasm. The conclusions of the 
acquiescence scales literature 

cannot be readily generalized to 

the issue of acquiescence as a 
response set tending to invalidate 
interview data. 


Summary and Conclusions 
There is a striking body of 


evidence attesting to the fact that 
interview data can be invalidated 
by a tendency on the part of re- 
spondents to give agreement re- 


_$ponses independently of the con- 


tent of the question. The studies 
using content-free measures suggest 
a possible mechanism for this 
effect; when respondents lack more 


- substantive bases for responding, 


they tend to prefer agreement re- 
sponses. The data drawn from 
situations where the correct answer 
is known support this conclusion 
and suggest additionally that an- 
sswers to false items are more valid 
than answers to true items. The 
most impressive data on acquiescence 
come from item-reversal studies. 
These studies demonstrate clearly 
that acquiescence can operate 
powerfully enough to seriously 
reduce the validity of interview 
responses. -Also, item reversal 
studies suggest that black respon- 
dents, less-educated respondents, 
and less intelligent ‘respondents 
may be particularly likely to 
acquiesce. These latter two sub- 
ject variables zre, of course,'of 
direct relevance to our efforts 

to‘ interview mentally retarded” 
respondents. Finally, data from 
individual difference measures of 
acquiescence have som. indirect 
relevance to the present study. In 
short, acquiescence is potentially 
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“sponse effects. 


views, and the data suggest that 
the problem may be accentuated in 
interviews with mentally retarced 
populations. 


Issues of Self-Presentation 

Many investigators have sug- 
gested that distortion of responses 
may occur when the respondent ‘seeks 
to portray himself of herself in a 
particular way. Sudman and Bradburn 
(1974) refer to such response effects 
as arising from "issues of self 
presentation." Within the personality 
assessment literature, much atten- 
tion has been given to the influ- 
ence of social desirability on re- 
sponding. That is, respondents may 
tend to endorse items perceived as 
socailly desireable and to avoid 
endorsing items preceived as so- 
undesirable, without re- 
guard to the "true answer." Thus, 
respciise patterns would reflect the 


. perceived social desirability of 


the items, and not the actual behav- 
iors or attitudes of the respon- 
dents. 
raise the possibility that items of 
an extremely personal nature might 
threaten the respondent and alter 
responses. 


Edwards (1957) presented data 
suggesting the importance of social 
desirability as a source of re- 
Subjects’ scores 

on personality scales frequently - 
correlate very highly with the 

rated social desirability of scale 
items. Edwards also argued that 

the correlation between two measures 


- frequentiy reflects the degree to 


which they have similar properties 
on the dimension of social desir- 
ability. 


Nevertheless, Sudman and Bradburn 
conclude that response effects 
attributed to threat and social 


Sudman and Bradburn (1974) also 


desirability are generally small. 
The interaction of self-presenta- 
tion variables with other variables 
may, however, be important. For | 
example, respondents are more likely 
to present themselves favorably when 
interviewed face-to-face than when 
afforded the relative privacy of 

a self-administered questionnaire. 
Thorndike, Hagen, and Kemper (1952) 
compared 500 self-administered and 
500 face-to-face administrations of 
an inventory on psychosomatic symp- 
toms. The respondents to the self- 
administered form reported some 15% 
more psychosomatic’ symptoms than did 


respondents to the face-to-face in- 
When Knudsen, Pope, and Irish 


terview. 
(1967) compared women's responses to 
face-to-face and self~administered 

interviews regarding the permissibil- 
ity of premarital sex, 20% of respon- 
dents to face-to-face interviews said 


it was all right to have premarital sex, 


as -opposed to 31% of the respondents 
to the self-administered form. In 
general, the magnitude of the effect 
due to method of administration in- 
creases as the probability of a so- 
cially desirable answer becomes 
greater. 


As has been mentioned already, ques- 
tion structure also interacts with . 
self-presentation issues to produce 
response effects. Closed-ended ques- 
tions appear to increase response 
effects due to threat or social de- 
sirability. 
action occurs between social desir- 
ability and respondent and inter- 
viewer characteristics. Sudman and 


Bradburn (1974) found response effects 


differ as a function of the: race and 
sex of both interviewer and respond- 
ent, depending on the degree to 
which an interview activated concern 
with self-presentation. 


In short, issues of self-presentation 
in themselves have small response 
effects, but interact with other 


Furthermore, some inter- 


to 


variables to produce important re- 
sponse effects. `; In particular, 

. method of administration interacts 
with social desirability; face-to- 
face interviews involving a strong 
social desirability component re- 
sult in considerably less admission 
of .information placing the respon- 
dent in a negative light. This find- 
iny may be important in interviews 
with mentally retarded persons, since 
these interviews are of necessity 
face-to-face. Mentally retarded re- 
spondents may tend to report atti- 
tudes. and behaviors which they be- 
lieve will meet approval, especially 
if they perceive the interviewer as 
having power. Social desirability 
also interacts to some degree with 
question structure and with the race 
‘and sex of’ the interviewer’ and re- 
spondent. l 


Behavioral. vs. Attitudinal 
Information 

One can distinguish two types of 
information sought in interviews. 
The first type is behavioral infor- 
mation, sought through such questions 
as, "In what year were you born?" 


The distinguishing characteristic of 


such information is that it can in. 
principle be verified by reference 
to outside sources. Attitudinal in- 
formation, on the other hand, in- 
volves the subjective valuations of 
‘the individual (e.g., "Do you like 
your current job?). Such questions 
have no external referent; that is, 
they cannot, even in principle, be 
verified. A ' 
There is some suggestion in the 
survey literature that attitudinal 
information is more subject to re- 
sponse effects than is behavioral 
information. However, because at- 
titudinal information has no exter- 
naľ standard of accuracy, the mea- 
surement of response effects for 
behavioral and attitudinal 


information differs fundamentally. 
Thus, response effects for the two 
types, of questions cannot be di- 
rectly. quantitatively compared, and 
no general conclusion as to the 
relative susceptibility of the two - 
types of questions to response l 
effects is easily drawn. 


Availability of Information 

The availability to the indi- 
vidual of the “information sought 
may have powerful effects on xe- 
sponse validity. That is, if in- 


‘formation is not very salient or is 


for some reason difficult to recall, 
the respondent's answer is likely > 
to be biased. 


Sudman and Bradburn (1974) showed `“ 
that response effects differ greatly 
according to-the length of the recall 
period involved in r’ porting behav- 
ioral information. Respondents tend 
to overreport behaviors for short 
recall periods and to underreport 

as length of recall increases. 


“Length of recall period also inter- 


acts with,severai other factors. 
Response effects increase substan- 
tially for individuals over 55 as 


the length of recall increases. 


Face-to-face interviews typically 
reduce response errors due to poor 
recall by reducing omission errors, 
although face-to-face interviews 
may also stimulate overreporting. 
Threatening topics increase response 
effects for longer recall periods 
due to omission. Also, open-ended. 
questions have been found to yield: 
somewhat' weaker response effects 
over increasing recall periods 


“than do closed-ended questions. 


Saliency of the information re- 

quested might be expected to have 
an important effect on responses. 
Surprisingly, Sudman and Bradburn 


found that saliency of questions 


had no general effect on behavioral 


items. However, low saliency in- 
creased response effects for atti- 
tudinal items. The authors specu- 
lated that the difficulty of mea- 
suring saliency might have been re- 
sponsible for the lack of a strong 
effect. 


In short, long recall periods in- 
crease response effects. Length 

of recall interacts with several 
other task variables in producing 
this effect. No general effect 

due to saliency of information 

was observed, although low sali- 
ency increases response effects for 
attitudinal items. 


Methou of Administration 

Analysis of response effects 
due to method of administration 
essentially involves a comparison 
of face-to-face with self-adminis- 
tration. Those effects are gen- 
erally best int. rpreved ‘as arising 
from interaction with other task 
characteristics. Most notably, 
as mentioned above, respondents 
are. far more willing on self-ad- 
ministered interviews to give in- 
formation placing them in an unfavor- 
able light. In fact, most of 
the response effects attributable 
to method of administration pro- 
bably arise from the fact that face- 
to-face interviews intensify for the 
respondent those issues of self-pre- 
presentation raised by the content 
of the interview. 


INTERVIEW VARIABLES 

The biasing effect of inter- 

` viewer characteristics on survey 
responding is the most widely re- 
searched area in the response ef- 
fects literature. Far more effort 
has been devoted to this topic than 
to the assessment of response ef- 
fects due to task variables. Never- 
theless, Sudman and Bradburn (1974) 


conclude that response effects duc 
to interviewer. characteristics are 
generally less important than those 
due to task variables. When inter- 
viewer variables exert important ef- 
fects, it is usually in the context 
of highly specific relationships 
among interviewer characteristics, 
respondent characteristics, and the 
topic of the interview. 


Sudman and Bradburn also found that 
the social status of the interviewer 
could interact with task and respon- 
dent variables to engender response 
effects. Katz .(1942) for example, 
found that working-class inter- 
viewers obtained more pro-labor 
responses, particularly from union 
members, than did middle-class in- 
terviewers. On war issues, middle- 
class interviewers obtained more 
interventionist responses than did 
working-class: interviewers. The 


finding may be of importance in 


interviewing mentally retarded per- 
sons, Particularly where the respon- 
dent might regard the interviewer 
as having authority. 


Sudman and Bradburn (1974) found no 
general effect due to sex or race of 
the interviewer, or,to any other com 
bination of interviewer character- 
istics. The only exception to 

this general lack of interviewer 
effects pertains to "don't know" 
answers. Fragmentary evidence sug- 


_ gests that interviewers over 50 


get more "don't know" responses, 
that interviewers of higher social 


- class and with more experience get 


fewer "don't know" responses, and 
that the “don't know" rate may be 
higher for female interviewers. 


Interactions of Interviewer 
Characteristics with Other Variables 

There are .some interactions be- 
tween interviewer characteristics 
and method of administration, 
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question structure, and self- 
presentation issues. 


_The. most notable response effects 
attributable to interviewer vari- 
ables occur when there are specific 
combinations of interviewer and re~ 
spondent characteristics and topic 
of interview. For example, Schuman 
and Converse (1971) conducted 500 
interviews in black households, 
using both black and white inter- 
viewers. Response effects due to 
interviewer race were large for 
items having a definite racial con- 
tent, especially items involving 
militancy cr hostility toward whites. 
In response to the question "Do you 
personally feel you can trust most 
white people, some white people, or 
none at all?", 35% of respondents 
replied "trust most whites" when in- 
terviewed by a white person, aS op- 
posed to only 7% when asked by a 
black interviewer. Schuman and Con- 
verse observed little or no effect 
due to interviewer race on questions 
not involving militancy on hostility 
to whites. This illustrates the 
general conclusion that response 
effects attributed to interviewer 
race or sex are highly specific to | 
the interview topic. 


_In summary, larger response effects 


tend to occur when the interviewer 

iS young or inexperienced. The most 

notable response effects occur, how- 

ever, when the topic of the inter- 

view is such that some combination of 
interviewer and respondent character- 
istics may bias respondent's answers. 


RESPONDENT VARIABLES 

l It seems likely that some types 
of respondents are more prone than 
others to giving biased answers. 
Sudmand and Bradburn (1974) found no 
general response effects associated 
with the sex, race, or age of the 
respondents. They did find that the 
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years or less of school. 


largest response effects tend to 
occur for respondents with eight 

This re- 
sult does not refer to poorly edu- 
cated adults but to children who 

are still in school. It is not clear 
what characteristics of school chil- 
dren make them -specially « :sceptible 
to response effects, but tais finding ' 
may have important implications for 
interviewing mentally retarded per- 
sons. Mentally retarded individuals 
are both "young" in the sense of men- 
tal age and uneducated, in that they 
do not progress normally through the 
educational system. This, if it is 
either Low mental age or lack of edu- 


‘cation that exaggerates response ef- 


fects in school children, we may ex- 
pect that mentally retarded indivi- 
duals will be particularly suscep- 
tible to response effects. i 


“Sudman and Bradburn also report some 


response effects due to particular 
combinations of respondent and task 
variables. Respondent variables in- 
teract to a minor extent with level 
of threat, method of administration, 
structure of questions, length of 
questions, and the possibility of 

a secially desirable answer. Two 
respondent-task-interactions, already 
discussed under task variables, are 
of particular relevance to our re- 
search. First, response effects for 


-closed-ended questions (i.e., over- 


reporting) are particularly high for 
elementary school students. Second, 
various response effects engendered 
by long questions are particularly 
strong for respondents with high 
school education or less. 


In short, elementary school students 
are especially susceptible to re- 
sponse effects. This effect is ın- 
creased by the use of closed-ended 
questions. Further, less educated 
respondents tend t- be more strongly 
affected by long questions. 


Implications for Interviewing 


Ketarded Persons : 

© What do these two very different 
bodies of literature, one on language 
and communication skills of the men- 
tally retarded and the other on re- 
sponse effects in survey research, 
tell us about what to expect when we 
interview retarded individuals? 
Although we have discussed specific 
implications along the way, we would 
like to emphasize the following ob- 
servations here: 


1. There has been almost no re- 
search, prior. to ours, specifi- 
cally examining the performance 
of retarded persons in inter- . 
view situations. Thus it must 
be recognized that the litera- 
ture we have reviewed here is 
only indirectly relevant to our 
questions of interest and that 
any expectations based on that. 
literature may or may not be 
borne out when retarded indi- 
viduals are actually inter= 
viewed. 


2. As a general rule, we can ex- 
pect retarded persons to behave 
linguistically much like 
younger persons of normal intel- 
lectual functioning. For the 
most part, they can be expected 
to perform like children of com- 
parable mental age. However, at 
times they may perform below the 
level one would expect on the 
basis of mental. age, and occa- 
sionally (e.g., with respect to 
vocabulary size) they may exceed 
such expectations. Such delays 
in language development repre- 
sent, quantitative differences 
between retarded and normal-IQ 
individuals, and thus no need ` 
to shape interviews in accord 
with unique characteristics 
of retarded persons. 


Because language development is 
generally related to mental age, 
little can be expected of re- 
tarded adults who are profound- 
ly retarded or retarded chil- 
dren who have very low mental 
ages. Certainly interviewing 
would appear to be unfeasible 
with persons whose mental ages 


` are below three; on the other 


hand, answering questions - 
appears to be well within the 
cognitive and linguistic capac- 
ities of those with mental ages 
above seven or eight. While 
mental age can be viewed as . 
the most important considera- 
tion, other factors may also 
influence how well an individ- 
ual can handle the demands of 
an interview. For example, 
expectations may be somewhat 
lower for Down's Syndrome chil- 
dren and for those who are in- 
stitutionalized. 


Another factor with major im- 
plications for interviewing 
retarded persons is their high 
rate of speech and. hearing dis- 
orders. Even when mental age 
is relatively high, speech 
defects and disorders may, make 
it difficult for the interviewer 
to understand what the inter- 
viewee is attempting to say. 
More crucially, the high preva- 


- lence of hearing disorders 


among the retarded suggests 
that some screening for hearing 
defects might be necessary or 
that alternative (nonverbal) 
communication systems might be 
necessary with persons who have 
significant hearing losses. 


Studies of semantic development 
among the mentally retarded 
point to the need for care in | 
wording questions. Most 
notably, concrete wording 
should be favored over abstract 
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wording, and commonly used words 
should be preferred over infre- 
quently used words. Studies of © 
the cognitive development of 
young children can serve as a 
useful guide in deciding what 
types of concepts might be too 
cognitively advanced for many 
retarded persons. 


Similariy, the structure of 
questior.s asked of retarded | 
persons should be kept as simple 
as possible. Questions with 
dependent clauses, passive con- . 
structions, negatives, “and so, 
because they are transforma- 
tionally complex, can be ex- 
pected to be difficult to 
understand. To be preferred 
are questions that are based 

on simple active declarative . 
"kernel" structures and that are 
as short as possible. Studies 
of question comprehension and 
production among normal children 
suggest a regular developmental 
sequence that can serve as a 
guide to constructing questions. 
For example, yes-no, what, and 
where questions can be expected 
to be easiest for retarded ~~ 
persons to understand and 
answer, while who, how, and 

why questions, and especially ` 
when questions, appear to re- 
quire a higher level of cogni- 
tive development.- 


Retarded individuals, like nor- 
mal children, can generally be 
expected to understand more 
advanced linguistic forms. than 
they can produce on their own. 
As we have noted, the implica- 
tion of this for designing 
interviews is that structured 
question formats may prove 
easier to answer than open-ended 
formats because they reduce the 
demands on the interviewee ‘to 
compose and express lengthy ` 
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answers. Questions should still 
be kept as simple as possible to 
aid comprehension, but anything 
that would make answering easier 
seems particularly advisable 
(e.g., asking respondents to 
point to a response, give a 
single word answer like "yes" 

or "no," or select one of two 

or three options). 


Whereas the_ literature on lan- 
guage and communication among 
the mentally retarded suggests 
that they may have difficulty 
understanding and answering 
questions, the literature on- 
response effects in survey 
research with the general popu- 
lation suggests that what they 
say may not be valid. This re- 
search has identified numerous 
ways in which answers are af- 


' fected by the nature of the 


interview task or the Specific 
questions asked, charactertistics 
of the interviewer, and charac- 
teristics of the respondent. 
There is no reason to believe 
that, mentally retarded inter- 
viewees are immune from such 
effects. Indeed, there is 
reason to believe that their 
answers might be especially 
influenced by such factors as 
how a question is worded or 
whether it is closed-ended or 
open-ended, for less educated 
members of the general popula- 
tion as well as young children 
have been shown to be especially 
susceptible to response effects. 


The response effects literature 
does not always suggest that 
one way of asking a question is 
preferable to another; often 
studies only demonstrate dis- 


,crepanciés between the answers 


that are given. However, this | 
research does suggest the wis- 
dom of avoiding. questions that 
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contain leading or emotionally 
laden wording; that are unusu- 
ally long and complex; and that 
encourage respondents to pre- 
sent themselves in a socially 
desirable manner. Acquies- 
cence, or the tendency to agree 
regardless of question content, 
would appear to be an especial- 
ly dangerous threat to the 
validity of answers given by 
retarded persons. 


Finally, the literature on 
response effects in survey 
research reminds us that any 
interview is a social situation 
affected by characteristics and 
behaviors of both the inter- 
viewee and interviewer. The ° 


. answers given by mentally 


retarded persons may bé 


affected not only by their 
characteristics but also by 
such. factors as the inter- 
viewer's age cr the amount of 





experience cr sot has. 
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characteric.icS thuc are -sS- 
pecially likely to yield in- 
valid answers. While our - 
research did not examine the 
effects of interviewer charac- 
teristics, we did look care- 
fully at response biases in 
the answers given by retarded 
interviewees, attempting’ to 
determine how both the way the 
question is asked and the IQ 
of the respondent affect the 
validity of what is said. — 
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The project was designed to ad- 
dress the following questions about 
interviewing retarded persons: 


1. To what extent can retarded 
persons respond to questions 
in an appropriate fashion, 
and what factors affect their 
responsiveness? 


2. How reliable are such respon- 
ses, in the sense of being 
consistent over short periods 
of time? 


3. How valid are such responses, 
in the sense of being free of 
systematic biases and agree- 

i ing with information provid- 

i ed by parents or caretakers . 

or documented in records? 


! 4. What types of questions 

|; . appear to optimize respon- 
© giveness, reliability, and 
validity? 


These guiding questions were explored 
through a series of five studies, 
each focused on a subsample of the 


Chapter 3 


METHODS OF THE STUDY 


retarded population and each having 
its own priorities. and basic. design. 
After describing the process of 
questionnaire development, we will 
provide an overview of the individual 
studies and of the methods employed 
in all of them. 


Design of Interview Schedules 


In early discussion with repre- 
sentatives of the President's 
Committee on Mental Retardation, the | 
scope of topics to be asked about was 
delimited. Three major topics were 
identified: living circumstances, 
familiarity with and utilization of 
services, and opportunities for 
decision-making. As a result, the 
interview schedules initially çon- 
structed typically included questions 
about basic characteristics of the 
respondent, major day activities, 
extent of involvement in other ac- 
tivities, nature of the residential: 
setting, training currently being 
received, and independence in daily 
decision-making. 
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Interview schedules used previously, 
including ones used in the Center's 
own studies of deinstitutionaliza- 
tion, were reviewed to generate 
potentially useful questions and 
ideas for alternative question phras- 
ing. Pilot interviews were devel- 
oped during the first part of 1977, 
and they were tested on a small 
sample of adults, both institution- 
alized and living in the local com- 
munity, in May, 1977. Extensive 
revisions and analyses of individual 
items were then conducted, and the 
pilot interview schedule was sent to 
PCMR for review. After further re- 
finement, questionnaires were con- 
structed for use ‘in the first 
interviewing venture with institu- 
tionalized children and adults in 
August, 1977. In these studies, - 
which addressed the reliability 
question, some questions were simply 
repeated in exactly ‘the same form 
on two interview schedules adminis- 
tered six to eight days apart. In 


_ addition, in these studies as well as 


in later ones, a major effort was- 
made to include alternatively struc- 
tured or worded questions on the 


same topics in order to test the : a 


relative merits of various ques- 
tioning strategies. For example, in 
asking about decision-making, we ` 
tested these two alternatives: "Who 
decides what clothes you put on in 
the morning?" versus "Is it up to 
you what clothes you put on in ‘the 
morning?" 


It should be noted that while some 
effort went into establishing the 
content of the interview schedules, 
it was minor in comparison to the 
effort that went into developing 
workable questions and alternative 
question forms. In other words, we 
were not overly concerned with devel- 
oping a standard. protocol to fully 
address the questions of living 
circumstances, services, and \ 


- were interviewed. 


A 


aecision-making. Instead, while 
using these topic areas as a context, 
we concerned ourselves primarily 
with exploring feasible interviewing 
strategies. On occasion, we included 
questions which we expected to 

fail simply to document that certain 
approaches are relatively unworkable. 
On occasion, too, we deviated from 


_the three major topic areas in our 
. quest for understanding of interview 


behavior. Finally, we did a con- 
siderable amount of alteration of 
questions from study to study, re- 
fining questions that had been used 
before and introducing new questions 
appropriate to the samples under 
study, so that, in the end, the 
various samples we interviewed 
responded to very few of the same 
questions. Appendix A, by presenting 
many of the questions that were in- 


cluded in the various interview 


schedules, offers a more concrete 
illustration of the kinds of ques- 
tions which were asked. 


The Five Study Samples - 
Because the study was explora- 
tory in nature, its design evolved 
over time, but from the start there 
was an interest in testing the feasi- 


. bility of interviewing with different 


segments of the mentally retarded 
population. The five studies col- Ek 
lectively involved both children 

and adults, in both institutional 

and community settings, as well as 
identified significant others l 
(parents, attendants, and advocates). 
In all, 180 mentally retarded persons 
Since some parti- 
cipants were administered repeated 
interviews, the total number of 

client interviews held was close to 
320, and in addition, 210 interviews 
were conducted with significant 
others. The five studies and their 
samples are described on the following 
page. . 


INSTITUTION CHILDREN 

The study of institutionalized 
children focused on’52 children, age 
12-16, who were residents Of a sin- 
gle state institution for the retard- 
ed: 20 severely retardéd by 
Stanford-Binet cutoffs (IQ 20-35), 
16 moderately retarded (IQ 36-51), 
and 16 mildly retarded. (IQ 52-68). 
A list of eligible participants was 
drawn by random sampling of the 
facility's files, and provision of 
consent and availability for inter- 
view then determined which eligible 
subjects (those in the right age and 
IQ range) were actually interviewed. 
Two interview forms were used, with 
some questions repeated and others 


phrased differently on the two forms, : 


and with the forms administered 
approximately a week apart. Through 
counter-balancing, half cf the 
children received one form first 
and half received the other, half 
were interviewed by one interviewer 
first and half by the second, and 
each child was then administered the 
other form by the other interviewer, 
as shown in Table 3.1. A cottage 
attendant familiar with each subject 


was also interviewed using an inter- ` 


view schedule which included most 
of the same questions asked of res- 
idents (with "you" changed to " he" 
or "she"). : 


I 


INSTITUTION ADULTS 

The study of institutionalized 
adults was identical in design, but 
involved adult residents of the same 
institution, including profoundly 
retarded persons. A total of 58 
adult subjects were ultimately 
interviewed: 16 profoundly retarded, 
16 severely retarded, 16 moderately 
retarded, and 10 mildly retarded. 
The interview schedules used with ` 


< this. group were longer and more 


intricate than those used with 
any other groups, and again 


attendants were asked most of the 


same questions. 


COMM‘. *: [TY CHILDREN 

The third study focused on 
children in the 12-16 age range; 57 ` 
in all, 18 severely retarded, 19° 
moderately retarded, and 20 mildly 
retarded; none of whom were institu- 
tionalized. Use of the 12-16 age 
range in both this sample and the 
institution children sample made 
them comparable in approximate mental 
age (the five- to eight-year old 
range). Students at the lower IQ 
levels,were drawn from a school for 
the trainable mentally retarded, 
while mildly retarded subjects were - 
drawn from other special education 
programs in the same city. Informed 
consent from parents, obtained by 


en ` 


Table 3.1: 





lst Administration 
4 of sample: 
3 of sample: 
4 of sample: 


+ of sample: 





Form A, Interviewer 1 
Form A, Interviewer 2 
Form B, Interviewer 1 


Form B, Interviewer 2 


Study Design 








2nd Administration 
-Form B, Interviewer 2 
Interviewer 1 


Interviewer 2 





Interviewer 1l 


home delivery of forms by children, 
determined which eligible subjects 

. were in the final sample, thus 
making it impossible to determine 
the extent to which the sample was 
representative. This study did not 
deal with the interviewer-reinterview 
reliability issue, but instead tested 
several new questions appropriate to 
community residents and several 
alternative question formats on the 
same topic. Each subject's parent 
or guardian was also interviewed. 


The original design of the project 
-cCalled for extending the research to 
a fourth major group of interest: 
adults in the community. However, 
by th. time that final phase was 
being planned, we had had an oppor- 
tunity to examine data from the 
first three samples and decided to 
alter the plan. It had become 
apparent that interviewing severely 
retarded persons and interviewing 
more highly verbal, mildly retarded 
persons were two different under- 
takings. On the one hand, we still 
wished to interview adults living in 
the community but felt that it might 
be possible with such a group to t2y 
out more unstructured questions and 
complex topics. On the other hand, 
we felt a need to search further 
for techniques which could be suc- 
cessfully employed with low verbal 
“persons. As a result, two additional 
studies, far different in samples and 
methods were’ conducted. 


_ COMMUNITY ADULTS oe 000 

To, facilitate identifying a di- 
verse sample of adults living in the 
community, we made contact with*the 
Association for Retarded Citizens in 
another city in Texas and requested 
their cooperation in the interviewing 
study. Letters and informed consent 
forms were mailed to. ARC-based citi- 
zen advocates, who were to indicate 
whether, they would be interested in 
being interviewed themselves. They 


. interviewed. 


v 


were albo asked to explain the proj- 
ect to their proteges and determine 
if they too would be willing to be ș 
Unfortunately, of the 
70 some letters mailed, very few were 
returned, and even with follow-up 
phone calls informed consent was ob- 
tained and interviews could be 
scheduled with only 13 mentally 
retarded participants and their 13 
advocates. The retarded participants, 
whose exact IQ scores weie not 


obtained, were generally, in the mild 


or moderate range of retardation and 
represented a highly verbal group. 
Interviews with them were conducted 
by a single interviewer and were 
tape recorded. The study, which 
yielded long and enlightening tran- 
scripts, provided an opportunity to 
use unstructured probing techniques 
and to explore difficult areas such 
as satisifaction with services and 
reactions to being, or beiny con- 
sidered retarded. While we will 
not provide detailed findings in 
view of the small sample size, we 
will occasionally quote from the 
transcripts to illustrate processes 
and problems in interviewing. 


REINTERVIEWING OF LOW VERBAL CLIENTS 

The final study in the series 
was designed to further éxplore 
techniques of eliciting information 
from low verbal subjects. It in- 
volved reinterviewing institution- 
alized children and adults who had 
been largely, but not totally, un- 
responsive in the previous institu- 
tion studies. The 29 subjects ranged 
in IQ from 20 to 48 and were in the 
lower half of the distribution of 
responsiveness, but had been able to 
answer at least one question in the 
first interviews. The phenomenon of 
acquiescence was explored further in 
this study and new approaches using 
pictures were tested out. Charac- 
teristics of the samples are sum- 
marized in Table 3.2. , 
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Institution adults 


Community adults 


Access to Subjects 

Since gaining permission to con- 
duct a study and gaining consent 
from potential participants are part 


“of any interviewing study, a word on 


the procedures used in the present 
study is in order. We elected to 
make contact with particular agencies 


or facilities, each of which had its - 


own procedures which we were then 
obligated to follow. If we had been 
seeking a representative sample of 
mentaliy retarded persons of all 


‘ ages and circumstances, the proce- 


dures used would have been far more 
elaborate and difficult to implement 


than the procedures we actually used. 


_ To compose the institution samples, 


ar approach was made to a facility in 
Arkansas with which the Center had 
had pr- oùs relationships. A pro- 
posal wus submitted for review and 
approval by the body charged with 
determining the appropriateness of 
research proposals affecting clients 
in the Arkansas MR Services system. 


Although it did not at first appear 


that ‘consent from all parents would 
be necessary, a decision was made 


that such consent would be necessary. `, 
‘As a result, a trip that was to have — 


been the data collection trip turned 


ee 


‘Table 3.2: Characteristics of Samples 


IQ 


ee 


Institution Children 


(N = 52) 


(N = 58)* 


Community chiidren 


(N = 57) 


= 13) 


Reinterviewed institution 
sample 


™ = 29) 


*The entire adult sample consisted of 58 persons, 16 of them with IQs from 0-19. 


n EE 


SD M. SD 

















However, 
as they were generally uninterviewable, and as most analysis focuses on the severe through 

mild ranges of retardation, they are not included in these figures, which are instead based on.. 
42 participants in the severe to mild range of retardation. 
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“returned. 


into a lengthy process of gaining 
consent. Parents of children and 
adults potentially eligible for the 
study were called by project staff 
members in the presence of a witness 
and asked to provide their consent. 
Following this, letters were sent 

to those parents or guardians who 
had given consent verbally so that 
their consent could be provided in 
writing. In addition, each client's 
verbal consent was obtained before 
the interview was actually begun. 


In the study of community children, 
the planned research again had to 

be approved through normz.l mechanisms 
in the school system., In this case, 
there was a prohibition against the 
research project staff making direct 
contact with parents or children, so 
a designated staff member in the cen- 
tral administration was responsible’ 
for distributing explanation letters 
and consent forms to teachers of 
mentally retarded students in the 
specified age range and then collect- 
ing the forms which found their way 
home with the students and were 
Using this procedure, many 
forms were never returned, and when 
it became apparent that few mildly 
retarded subjects were being identi- 
fied, new rounds of forms were dis- 
tributed to teachers of the mildly 
retarded. 


Similarly, to constitute a community 
adult sample, we first had to obtain 
the endorsement of a local Associa- 
tion of Retarded Citizens. In this 
case, a project staff member was 
given permission to mail letters and 
forms Airectly to all citizen advo- 
cates and to directly receive the 
returns, and written client consent 
was also required. Again, many forms 
were not returned. In no study ‘was 
the rate of refusal high, but in 
those that involved return mailing 
of consent forms, low rate of return 
was the general rule. Unfortunately, 
this creates uncertainty about how 
representative those who actively 


provide consent are. The telephoning 
procedure -used to constitute the 
institution samples was highly effec- 
tive by comparison, but in view of 
the recent tightening of procedures 
for access to subjects, it may not 
continue to be acceptable, for it 
does involve a. facilicy's making 
available to outside researchers 
clients' names and addresses. The 
moral is that an interviewing project . 
of this nature must leave adequate 
time for obtaining permission to 
conduct the study and for gaining 
consent. There are special problems 
in making the nature of the study 
clear to mentally retarded people. 
Appendix B discusses some of the 


- current issues in obtaining research 


approval and informed consent. 
Appendix C illustrates how the 
present study was explained to men- 
tally retarded persons immediately 
before they participated and in- 
cludes. a sample consent letter to 
parents. 


Measuring Responsiveness 

‘An important focus of the project 
throughout was the ability of men- 
tally retarded persons to answer 
questions in a manner appropriate 


‘to the form and content of the ques- 


tion, whether or not the answer was 
"true" or valid. Responsiveness to 
the communication demands of ques- 
tions was measured in each study 
through a slight modification of a 
coding system originally developed 
by Sigelman and Werder (1975), which 
included nine categories: 


l. no response 
2. unintelligible response 
3. irrelevant response 


4. don't know, don't remember, 
not sure (when respondent can 
be. presumed to know) 


5. inadequate response (vague or 
uncodable) 
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request for clarification of the 
question (huh? response) 


refusal to answer 


minimally appropriate response 
(response which meets the formal 
demands of the question by pro- 
viding one relevant bit of in- 
formation, e.g., "yes" or nam- 
ing one item in response to an 
open-ended question) 


expanded response (response which 
provides qualifying or additional 
information beyond the formal de- 
mands-of the question) 


Interrater reliability for this coding 
system has consistently been estab- 
lished at .85 or higher. In all studies, 
the interviewee's first response to the 
question was coded in this manner by 


‘ the interviewer either during the inter- 
‘view or afterwards using notes on the 
_ response given. 


In the adult institu- 
tional sample, responses to a second 
asking of the question when the first 
response was inappropriate were also 
coded in this manner. In the children's 
instituticnal sample questions were 
‘asked only once, while in all other 
Samples questions were asked a second 
time, if necessary, in exactly the 
same foxm. While .responsiveness to - 
the ficst asking formed the measure of 
sy EA in all studies, dings 
the contents of responses were bas- 
ed on whichever response, first or sec- ` 
ond, was at least minimally appropriate. 
The number of responses coded 8 or 9 
(minimal or expanded) was divided by 
the number of questions asked to form 
individual responsiveness scores. To 
form difficulty scores for given ques- 
tions, the number cf interviewees whose 
responses were coded 8 or 9 was divided 
by ithe number of persons asked the ques- 
tion. 





“In all studies a simple screening pro- 


cedijre was used when a subject's abili- 
ty tlo engage in an interview was sus- 
pect. The procedure consisted of three 


simple questions ("What is your name?", 
"Is your name (name)?", and "Can you un- 
derstand me?), each of which was asked 
a second time if no initial response 
was obtained. If any response (word, 
sound, gesture) to one of these questions 
was given, a subject was administered 
the full interview. Those who would 

not respond at all were defined as to- 
tally unresponsive to all questions in 
the full interview, even though it was 
not actually administered. 


Analyzing Reliability and Validity 

In all studies contingency tables 
were analyzed to examine issues of 
reliability and validity. In compar- 
isons Of subjects' responses to a 
first and second asking of the same 
question or to alternatively phrased 
questions on the’same topic, contin- 
gency tables indicated the percentage 
of the re.ponding sample providing 
the same answers on both occasions. 
Similarly, subjects' responses can be 
cross-tabulated with significant 
others' responses to the same ques- 
tions. At the same time, cells 
representing inconsistency between 
paired responses can be examined to 


“identify particular patterns of 


response error. The sample contin- 
gency table presented in Table 3:3 
will give an illustration of the 
method used. It presents proportions 
(rather than actual nvmbers) of 
various pairings of client and 
significant other responses about 
church attendance. If we look at 

the marginals of the table, 
note that there are discrepancies 
between client and. significant other 
responses, for 80% of the clients 
claim that they go to church while 
70% of the significant others claim 
that clients go to church. If we 
add the proportions of yes-yes and 
no-no combinations of response, we 
have an cverall agreement figure 
indicating that in 70% of the cases 
in which answers were paired, client 
and significant other agree. Ex- 
amining the cells representing dis- 


we would, 








oS | —— 





Table 3.3: Sample Contingency Table 











‘Significant other: 





Does client og to 
church? l 





ay. eement, we note that it is more 
common for the client to say "yes" 
and the significant other "no" than 
for the client to say "no" and the 
significant other "yes." In this 
instance, then, we have detected a 
systematic bias in responses that 
accounts for disagreement - one which 
probably reflects the acquiescence 
phenomenon with which we will con- 
cern ourselves later. , 


On the face of it, 70% agreement 
sounds fairly good. However, note 
that both clients and significant 
others were highly likely to answer 
"yes" to thi question, and that as a 
result, the 4ds by chance alone’ 

of obtaining many yes-yes combina- 
tions of answers is rather high. In 
other words, in tables like this one, 
we Cannot Simply assume that by "~ 
chance alone client and significant 
others would have agreed 50% of the 
time. Unfortunately, traditional 
statistical technig ies desi gre for 
use with contingency tables, s#easuces 
of strength of association =x) +s 
phi and gamma and lambda ana me<sures 
of statistical significance sych as 
chi-square, do not provide an” appro- 
priate means of determining how much 
a given agreement figure exceeds 





Do you go to church? 


‘chance. 


a aa 






what can be expected on the basis of 
However, Cohen (1960) has 
described a statistic called kappa 
which appears to suit this problem 
perfectly. l 


Because the project has involved 
hundreds of contingency tables, we 
have not calculated kappas for all 
tables, but we will make reference 

at various places in the report to 
kappa statistics, for they provide 

us with a’means of putting agreement 
figures in some perspective. The 

same logic that we have described 
here also applies if one is concerned 
with whether clients answer the same 
question the same way on. two occasions 
and whether their answers to two 
different forms of question are 
consistent. 

















l Footnote 
To illustrate kappa, note that the obtained proportion of agreement for Table 3-3 
is..7G. The proportion expected by chance is easily calculated from the marginals associated 
with the two cells representing agreement. The chance probability of a yes-yes combination 
is .56 (.70 X .80), while the chance probability of no-no combination is .06 (.30 X .20). 
Therefore the proportion of agreements expected by chance is .56 + .06 or .62, quite a bit 
higher than 50-50. Kappa is calculated as follows: 


k = Pobtained - p by chance , ý 
1l - p by chance 


In our example, 


Z .70 - .62 
-38 =. 


Very simply, this means that 21% of the cases represent agreement beyond agreement that one 
would expect on the basis of chance. Kappa ranges from -1.00 to +1. 00, a negative kappa indi- 
cating that disagreement is more likely than one would expect, a kappa around zero indicating 
that agreement is not very ‘different from what would be expected based on chance, and only a 
large positive kappa indicating agreement beyond chance probability. The statistical $ 
significance of a kappa can be determined by first calculating a standard error term, 


" 


o ko Pe 


NG - Pc) 


or in the example, assuming an N of 50, 


Oko = 


62 __ = .0326 = 
50(1 - .62) 
ok 2A, 
then Z = Yo 7 1.17 


_ By referring to a normal curve distribution table, we find that the probability of this z value 
is .12, approaching the conventional .05 level of probability but not large enough to be 
considered statistically significant or a systematic departure from chance probabilities. 
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- Most of the subsequent chapters 


of this monograph coricern the inter- 


view responses of mentally retarded 
persons. and their parents or care- 
“takers--the data of the project. 
However, we have gained, through 
year's of experience interviewing 


“ . mentally retarded persons, another 


type of "data," often subjective in 
nature, about the process of con- 


E ducting interview research with the 


mentally retarded. Since mentally 
retarded people are, after all, 
people, much of what we would ‘have 
to say is simply a matter of sound 
survey research design and practice. 
The literature on survey research is 
helpful as a source of general prin- 
ciples of interviewing applicable in 
studies of the mentally retarded. 

In Chapter 2, for example, we dis- 
cussed many of the variables such as 
Question wording, interview behavior, 
and subject characteristics which 
can produce error in any survey 
research project. Field interviewer 
manuals developed for surveys of the 
general population are also useful. 
For example, we have found the 


Interview::'s Manual developed by 
the Survey Research Center at the 
University of Michigan (1969) highly 
informative about.everything from 
sampling to closing the interview 
gracefully. 


on the other hand, we feel that there 
are some unique problems in con- 
ducting survey research with the 
mentally retarded. As later chapters 
will demonstrate, obtaining meaning- 
ful answers from low verbal persons 
is problematic and requires special 
techniques. In this chapter, we` 
focus on the logistics of interview-.. 
ing the retarded, drawing on the 
lessons we have learned through trial 
and error in hopes that others may be 
spared our errors. 


Designing Interview Schedules 
for the Retarded 


In any survey research project, 
the researchers presumably have an 
idea of what they are interested in 
learning and are guided by this ide: 
in developing questions. The meze 
specific the idea is the better; thus 


it is important as a first step to 
outline in some detail the topic 
areas of interest, the information 
goals. It is also important where 
possible to anticipate which topic 
areas may be difficult for mentally 
retarded persons. Much of cur data 
will pinpoint topics about which we 
had great difficulty obtaining in- 
formation. There were other topics 
we tended to avoid from the start 
because we anticipated problems. 
Generally, it has been our “experi- 
ence in previous interviewing re- 
search that many mentally retarded 
persons have difficulty with time, 
number, and money concepts (e.g., 
giving a chronological account of 
their whereabouts or their time 
allocations duriry the day, or in- 
dicating the number of times they 
have done something in the past 
month, or describing their incomes 
and sources of income). Words | 
such as "before," "since," and 
"after" are difficult (e.g., "Where 
did you live before that?"). We 
have also found it difficult to ask 
about experience with service agen- 
cies, as clients often view the 
agency as "Mrs. Brown" or "Mr. Tom's" 
and do not have = romplete under- 
standing of an aseusy and its func- 
tioning. We have also found it dif- 
ficult to ask hypothetical ques- 
tions which ask ‘the respondents to 
` imagine a different world from the 
one they currently experience. 
Some of these areas are problematic 
enough that, where possible, the 
information is probably best sought 
from nonretarded persons who know 
the subject well. In other cases, 
the process of developing questions 
will simply be more challenging and 
time-consuming. 


Surveys of the mentally retarded 
population will almost inevitably 
involve some branching in the inter- 
view schedule; that is, certain re- 
sponses wili dictate that succeeding 
questie .1 the topic be skipped. 
One must know enough about the kinds 


` tween work and nonwork. 


of persons one is to interview and 
their situations to be able to 
design meaningful subsections and 
anticipate exceptional cases to 
whom questions will not apply. 
However, there is often a risk 


. that the distinctions the research- 


ers think they are making will not 
be accorded the status of distinc- 
tions by retarded persons. For 
example, when asked if they work, 
many clients who are in sheltered 
workshops, work activity centers, or 
simply helping out their relatives 
on an informal basis will say "yes." 
The researcher may have sought a 
distinction between being competi- 
tively employed and bei. gy unemployed, 
but retarded interviewees may make 
quite a different distinction be- 
Further 
problems arise when clients do not 
distinguish between sheltered work- 
shops, activity centers, and so on. 
In fact, they may know these types 
of settings only by their local 
proper name. .Follow-up questions 
are often needed to make possible 
the kinds of distinctions that the 
researchers had in mind. 


Similarly, we have encountered many 
difficulties associated with ques- 
tion phrasing, such that respondents 
may not be making the same distinc- 
tions we choose to make between, 

for example, being able to do some- 


‘thing, doing it at least sometimes, 


doing it regularly, or being allowed ` 
to do it. The more prior knowledge 
one has of retarded persons and 
their views of the world, the more 
likely it is that questions will 

be sound. However, pilot testing is 
the best single method of checking 
to see that the questions are work- 
ing and determining whether addi- 
tional follow-up questions might be 
needed to clarify answers. 


There is a natural tendency in the 
question dev! ont phase for 
questions .O p vizferute. Me 
naturally e-"k&s moso specificity 


62 


about any area to be covered, but 
also tends to become sidetracked 
in fleshing out an area of question- 
ing. Having clear information 
goals as a guideline is one check 
against this tendency, but later 
editing may still be called for 
after pilot testing. It is our 
feeling that the interviews them- 
selves should not consume more 
than about half an hour or it be- 
comes difficult to maintain active 
attention. 


Our interviewing studies have each 
required two major forms besides 
a client interview schedule: a 
record “orm and a significant other 
interview. The record form is used 
to collect whatever information 
is available from ayency or facility 
files about demographic character- 
istics, IQ score, length of insti-- 
tutionalization, daily schedule, 
and so on. In our studies, parallel 
interviews with "significant others" 
(parents, caretakers, or others who 
know a client well) have’ typically 
asked almost all of the same ques- 
tions asked of retarded persons 
directly, excluding some subjective 
questions. This was done in order 
to explicitly examine the corre- 
spondence of answers.. In most 
studies, however, there are some 
types of information, as we suggested 
above, that are best gotten from 
` significant others or caretakers | 
for which the interviewer would at. 
least want to seek verification of 
answers given by retarded persons. 
In many studies, significant. others 
might also be used’as the major 
source of information about such 
things as clients' histories, cur- 
rent skill levels, and degrees of 
independence. Indeed, we find it 
hard to imagine an interviewing 
study which would not require at 
least a minimal amount of data from 
files or from knowledgeable informants. 
We found it best to interview such 
informants personally rather than 


have them fill out the questionnaire 
on their own, unless the questions 
were very simple and straightforward. 
The same principles of survey re- 
search applicable to retarded per- 
sons apply to their significant 
others, for their answers too are 
subject to bias, and, in’ some cases, 
they lack knowledge about certain 
areas. This is true even of parents. 


Pilot Testing ne 

Assume that careful planning 
has led to the development of an 
interview schedule. As noted above, 
much of the information in the 
present report is intended to guide 
the task of designing questions 
which can be answered reliably and 
validly by mentally retarded per- 
sons, so we will not discuss ques- 
tion design further here. What 
we want to stress is that what 
appears to be a good interview 
schedule may not turn out to be one 
in practice. Take the simple ques- 
tion, “Do you make your bed?" As 
our data will indicate, this yes-no 
question may be subject to acquies~- 
cence which inflates the number of 
interviewees reporting that they 
make their beds. Moreover, the 
question is ambiguous. It does 
not indicate, for example, where 
bed making takes place. Often in- 
stitutionalized, persons do some 
things at home that they do not 


. do in the institution and vice versa. 


The question also leaves open the 
question of extent of activity since 
a "yes" response may mean that the 
respondent has made a bed once or 
that he or she makes a bed every day. 
In constructing questions, one must 
adopt a highly critical attitude, 
actively looking for ambiguities 
and actively imagining potential 
misinter:retations of the question. 
It is helpful to imagine interview- 
ees in diverse situations to judge 
whether a question is. indeed appli- 
cable and meaningful to the whole 
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‘persons. 


. refining questions. Th 


‘sponse given is what 


Yange of persons to be interviewed.. 


Instead of thinking about the gen- 
eral rule, one does best thinking 

about all imaginable exceptions to 
the general rule. 


Imagination can carry one only so 
far, however. Pilot testing of 
interview schedules is, in our opin- 
ion, essential. A first step , 

may be role playing the interview 
with co-workers playing retarded 

This at least provides 

an estimate of the approximate 
length of the interview and iden- 
tifies questions which may have 
looked good on paper but do not 
sound good when they are asked. 

Those playing the role of inter- 
viewees can be asked to actively 
search for chances to misinterpret 
the question or raise questions 
about’ its meaning. The next step 
should be pilot interviews with 
retarded persons similar to the 

ones who will actually be inter- 
viewed. Ideally, these pilot inter-: 
views should be tape recorded, ` 
and the interviewer should feel 

free to rephrase questions on the 
spot‘if they do not seem to be under- 
stood. Such rephrasings ang the, 
responses to them are offten useful in 
‘interviewer 
ine if the re- 
e intervicwee 









can also probe to dete 


meant to say in caseg 
ty of response seems suspect. We have 
found that conductíng pilot interviews 


inevitably reveals problem areas that 


would otherwise go undetected: ques- 
tions that few can answer, questions 
that are greeted by puzzled expres- 


sions, questions that are quite consis- 


tently misinterpreted, and questions 


or questioning strategies that prove 


embarrassing for both interviewer 
and interviewee. In order to gather 
information about the validity of 
responses, it’ would also be useful 

to compare clients' answers to 
documented fact or selected responses 


.given by nonretarded informants in 


parallel interviews. 


where the validi- 


Interviewer Training 
It is our opinion that virtu- 
ally any sensitive and conscientious 


‘ person can become an effective inter- 


viewer of retarded persons. The 

more experience the interviewer has 
had with retarded persons, the 

better, however. We suspect, although 


“we did not encounter the problem 


ourselves, that some people, un- 
familiar with retarded persons, may 
not work out as interviewers if they 
have negative reactions and those 
reactions are noticed by: interviewees, 
In previous research, we actually 
encountered similar. problems with 
some persons experienced in mental 
retardation--usually agency staff 
who carried an authoritarian atti- 
tude into their interviews. Assum- 
ing that willing and able persons 
are recruited, the main tasks are 

to train them to use the interview 


- schedule and to standardize their 


behaviors as interviewers. 


Interviewers should, of course, be 
well-versed in the content of the 
interview schedule before they con- 
duct interviews. A serious pilot- 
ing effort can provide the setting 
for training. Where feasible, it 
is useful to have two interviewers 
record the answers given by pilot 
subjects so that the second inter- 
viewer can provide feedback on the 
first's approach and the data 
collected by the two can be compared 
for reliability. 


Interviewers not only have to be able 
to use the interview schedule 
confidently, but they must be well- 
versed in all aspects of conducting 
the interview. They should have a 
common understanding of any ques- 
tion rephrasing or probing procedures 
to be used; they should have a 
consistent approach to providing non- 
specific feedback to interviewees 

to maintain rapport and encourage 
answering; and they should establish 
in advance how to handle various 
problems that may arise. Research 
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discussed in Chapter 3 suggests 
that interviewer characteristics 
and behaviors can at times influence 
responses. Certainly if one inter- 
viewer is highly warm, supportive, 
and attentive while another goes 
through the motions in a cold and 
mechanical manner, one can expect 
very different results. There is 
also the risk that a warm and re- 
sponsive interviewer can become 

more mechanical after hours of back- 
to-back interviews. Thus, inter- 
viewer manner can be as important 

as interview content. 


Locating Interviewees 


and Scheduling Interviews 

In some types of studies in 
which subject's addresses have been 
obtained from an agency Or facility, 
simply locating the subject and 
arranging for an interview can be 
problematic. When one arrives at 
the supposed address and finds that 
the client\no longer lives there, 
one can check with neighbors, the 
telephone and utility companies, the 
Post Office, or the Police Depart- 
ment. City directories usually on 
file with the Chamber of Commerce 
can also be useful. Other potential 
sources of! information include local 
churches and small neighborhood 
grocery stores. The interviewer 
on a search should always explain 
his or her affiliation and purpose, 
but should avoid indicating that the 
persons being sought are mentally 
retarded. | As Edgerton (1967) has 
vividly shown, for example, many 
deinstitutionalized persons have 
struggled ko avoid labeling and 


` stigma; they would not appreciate 


having their “covers" blown 
by an insensitive interviewer. 

| 
Initial contact with an interviewee 
can be made either by telephone to 
set up an appointment or by going 
to the home or residential facility. 
Often cooperation is more likely 
if one goes directly to the home, 


fd 


but telephone contdct is more 
cost-effective and works weJl if 
the client has already prov .ded 
consent by mail. If telephone 


.- contact is made to schedule appoint- 


ments, it will still be necessary 

to introduce yourself and the study 
briefly when you arrive for the 
interview as time will have passed 
in the interim. Plenty of time 

for travel and unanticipated de- 
lays (often informal chatting with 
interviewees or their family) should 
be allowed between interviews. © 


Obtaining Consent 

In Appendixes B and C, we pre- 
sent additional information on re- 
search guidelines relevant to survey 
research as well as sample consent 
and explanation forms used in the 
present study. The points below 


- simply summarize the kinds of in- 


formation needed to make consent 
informed consent. There is a special 
burden on interviewers of the re- 
tarded to insure that the study is 
understood, and extra effort and 
patience may be required. The em- 
phasis in communications with retarded 
persons as well as their parents 

and care providers should be on 
simple, concrete, and even purposely 
redundant language. In obtaining 
consent for an interview, explain 

the following in simple, concrete 
language: 


1. Who you are 


2. Who the interviewer will be 
(if it is not you) 


3. For whom you are doing this 
/ 
4. Why you want to talk to them 


5. How long the interview will 
take 


6. What kinds of questions you 
will be asking a 


7. When you will be conducting 
the interview (approximately; 
schedule specifics based on 
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their schedules and desires 
as a matter of courtesy) . 


8. Where it will take place 
(Typically their own living 
situations put them most at 
ease, but it should be left 
open in case they prefer 
another place) 


9. Who else is involved--e.g., 
how you got their names, 
others you will be talking to 
about them, etc. 


10. How it will affect them (or 
the fact that it will not 
affect them if all informa- 
tion is to be kept confiden- 
tial) 


11. Who will see their answers 
and who will receive any re- 
ports that are based on the 
answers of many interviews 


12. How you will be asking for 
- their consent and what they 
need to do to give it 


13. That participation is volun- 
tary--that they do not have to 
be interviewed if they do not 
want to, that they can end , 
the interview at any time, 
and that they can skip ques- 


tions they do not want vo 
answer 


14. Whom to contact aid how to do 
so should they have any ques- 
tions 


Depending on the regulations for- 
access to subjects in effect, these 
points are either presented in a 
letter requesting consent to partici- 
pate or they are presented orally 

in person or by telephone, before 

the interview is to be conducted. 

Our preference is for the oral pre- 
sentation not only because it gives 


the client the opportunity to ask 


questions and clarify any misunder- 
standings, but also because the: re- 
turn rate’ on mailed consent forms 
is usually quite low. Moreover, 
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when written consent is obtained 

by mail, there is often a long time 
lag between the giving of consent 
and contact with the consenter 

for purposes of scheduling an inter- 


view. 


At that time, a rehash of 


the important aspects of the study 

is often required to remind the 
subject of what was agreed to earlier. 
If not previously obtained by letter, 
written consent should be obtained 
after oral consent is given. After 
consent is obtained, we reinforce 

the following points immediately 
before the interview begins: 


-1. 


The interview is not a test, 
and there are no right or 
wrong answers. 


Everything said will be kept 
a secret (assuming that is 
indeed the case). 


The interview will not change 
their lives in any way. We 
hope that it will help other 
people like them someday, but 
we are not going to do anything 
to help interviewees with 

any problems they might have, 
nor will we make them do some- 
thing they don't want to do-- 
like move or go to another 
school. 


Some of the questions may 
sound silly, but we need to 
ask all of them. (Our .sur- 
veys indeed involved blatantly 
silly questions as checks 

on acquiescence, but virtually 
any survey contains some ques- 
tions that make interviewees 
wonder. We use. the term 
"silly" rather than "dumb" or 
"stupid" because its conno- 


' tation is more favorable. 


We also point out other pe- 
cularities, e.g.) the fact 
that.some guestions are asked 
more than once.) 


They do not have to answer 
any questions they don't want 
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_ to, and can end the interview 
any time. 


6. If they need to stop, to get 
a drink or go to the restroom, 
they should feel free to say 
So. 


Interviews with retarded persons do 
not technically require parent or 
caretaker consent if the client is 
of age and has not been judged in- 
competent. However, it is a reality 
that parents and others who are re- 
sponsible for retarded persons have 
a stake in what happens to them. 
Whatever consent procedures are used, 
it is important to make contact with 
such significant others so they 

are assured of what you are doing and 
know that it will not affect the cli- 
ent's life or theirs or involve any 
risk of harm to the client. It is 
almost always necessary to make ap- 
pointments for interviews directly 
through such persons. as you will be 
in their homes or their residential 
facilities. Due to the difficul- 
ties that some retarded persons 

have with time concepts, such 
persons are also likely to be more 
aware of client's schedules than 
clients themselves are. The ma- 

jor point, however, is- that the 
position of a person responsible 

for the retarded interviewee’ should 
be respected despite the fact that 


‘their formal consent may not be 


required.. When parents and care- 
takers understand the study, they 
are put at ease and are more likely 
to accept your presence. We have 
encountered the most reluctance 

in some residential facilities 

where staff fear the interviews 

may put the facility in an unfa- 
vorable light. In one: interviewing 
study in which we were invcelved, 

a whole group of clients at one 
facility had to be dropped from 

the sample because houseparents were 
afraid that the study would "dig up" 
something. Anticipating these kinds 
of reactions beforehand, you can be 


A 


- client's interviews. 
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ready with answers to concerns that 
might be raised. 


Getting Ready to Interview 
We would not think it worthy o£ 

mention if it had not happened occa- 
sionally, but one can never do 
enough last-minuťe checking. One of 
„our interviewers, after numerous 
calls and visits, finally lined up 
an appointment for an interview, only 
to arrive, open her briefcase, and 
find that she did not bring the cor- 
rect interview schedule with her. 
Pencils break, mimeographed forms 
sometimes have missing pages, and 
whatever can go wrong often does. 


w . : 
When one is interviewing the retarded 


in their homes or in residential facil- 


ities, it is important to try to 
find a quiet place away from others 
in the household. We have repeatedly 
encountered difficulties when parents 
or caretakers attempt to take over 
and answer questions for the clients 
or react to, what they are saying. 

The best way to avoid this is to ex- 
plain the need for privacy and your 
interest in hearing what the client 
has to say, even if hétpr she may be 
giving factually incorrect informa- 
tion. Sometimes it is helpful just 
to tell the parent that many clients 
will want or expect their parents to 
answer some of the questions and that 
you need to avoid this if possible. 
Also, explaining, that the same ques- 
tions will be asked of them, if that 
is being done, alleviates their fears 
of the client giving out misinforma- 
tion. If you intend to give a ques- 
tionnaire rather than an interview 

to the significant others, it may be 
given to them to fill out while you 
are interviewing the clients. This 
will keep them occupied so they are 
not so likely to be invo`ved in the 
When the house- 
hold is particularly no!.sy and 
crowded or otherwise i:imical to a 

. successful interview, we have some- ` 
, times asked the client if .he or she 
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would mind sitting outside for the 
interview. Although parents must be 
allowed to sit in if they insist,- 
they may need to be reminded to let 
the client answer the questions. 

This arrangement is not as desirable 
as total privacy, but it is sometimes 
unavoidable. 


Relating to the Interviewee 
Although obtaining consent to be 


‘interviewed is obviously important, 


it is perhaps more important to move 
beyond mere consent and attempt to 
make the interview an enjoyable ex- 
perience. An interview should be a 
give and take situation. In other 
words, matena of focusing exclu- 
sively on one's own need to obtain 
information, be sensitive to the in- 
terviewee's needs. 
that many mentally retarded persons 
find the experience of having someone 
sit down individually with them and > 
listen to them rare and gratifying. 
They may want you to look at their 
pictures, listen to them talk about 
something unrelated to the inter- 
view, or meet their friends. Often 


-you can suggest that such things be 


done as soon as the interview is 


‘concluded, and this will satisfy 


them as well aş provide an incen- 
tive for continuing the interview. At 
other times it is best, particular- 
ly when clients want to tell you 
something important, to take a break 
on the spot and let them talk. This 
is why it is necessary to schedule 
more time than is required to do the 
actual interview. If an interviewee 
persistently wants to pull you off 


the course, politely steer him or 


her back to the questions, with 
something like, "That's interesting. 
Now I'd like to ask another ques- 
tion." i 


On occasion, a mentally retarded 
person will display inappropriate 
behavior in an interview. Our in- 
terviewers have encountered the ga- 
mut, from masturbation, to pulling 


» 


we have discovered 
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the hair of the interviewer, to 
getting up and wandering around the 
room, to simply carrying on a mono- 
logue totally unrelated to the in- 
terview. One of our interviewers 
simply ignored the fact that her in- 
terviewee was masturbating during 
the interview, but not all inter- 
viewers will find this quite sound 
approach of ignoring inappropriate 
behavior acceptable. If, as occa- 
sionally happens, a client poses a 
physical danger to the interviewer 
or to himself, terminating the in- 
terview is the obvious course of ac- 
tion. In response to most disrup- 
tive behavior, the best policy is to 
be firm but pdéliite in attempting to 
steer the client back to the inter- 
view as well as to reinforce, 
through smiling and words of encour- 
agement, paying attention and answer- 
ing questions. 


Perhaps the most. important word in 
interviewing retarded persons is re- 
spect’. Throughout history, mentally 
retarded persons have been treated 
as less than fully human individuals. 
Repeatedly they find themselves in 
situations where two nonretarded 
people talk about them as though 
they were not there instead of ask- 
ing their opinions directly. In 
scheduling an interview, in con- 
ducting it, in closing it, the in- 
terviewer should remember that the 
interviewee's needs come first, that 
he or she is helping you. It is not 
enough to think, "I'm going to help 
retarded persons through these in- 
terviews and {'1ll make them feel 
that their contribution is impor- 
tant." There is a major difference 
between this attitude and feeling 
that the interviewee actually is 
important and can help ycu. Re- 
tarded persons can detect this dif~ 
ference and are likely to respond 
much more openly to the extent that 
you give them reason to trust you 
znad come across as a person who is 
genuinely interested in them and 
their opinions, whatever they are. 
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For some mentally retarded persons, 
especially those who have left in- 
stitutions for the. community or 
whose present situations are in jeop- 
ardy, there may be a threat implied 
by the interview. In particular, 
formerly institutionalized persons 
and their guardians may worry that 
their answers will somehow result .in 
their being returned to the institu- 
' tion or being forced into other un- 
desirable living, work, or school 
settings. They may perceive the in- 
terviewer as anadult with power over 
« their lives. It it therefore essen- 
tial to emphasize strongly both the 
confidentiality of the information 
they will. be-providing and the fact 
that being interviewed will not 
change their lives in any way. Of 
course, if the project is such that 
it may affect their lives, they 
should be informed honestly of this 
during the initial explanation pre- 
ceding consent. Hopefully your hon- 
esty with them will help put them at 
ease and encourage honest answers. 


Sensitivity to the interviewee's 
needs during the scheduling and con- 
sent process is a start, but trust 
must also be nurtured throughout the 
interview as well. One way to do 
this is to provide nonspecific posi- 
tive feedback along the way. It is 
dangerous to praise or endorse spe- 
cific responses during the interview 
because you may inadvertently shape 
future responses. To provide nonspe- 
cific feedback, you might say, 
"That's really interesting," or "I'm 
glad you told me that." Smiles and 
nods after responses are even less 
specific and should be used frequent- 
ly. You should generally have an in- 
terested look on your face and avoid 
any expression of disapproval of what 
is said. It is best to put any ques- 
tions that can be considered sensi- 
tive near the end of the interview in 
hopes that by then a relationship of 
trust will have been fostered. How- 
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ever, avoid ending the interview on a 
depressing note, by returning to less 
sensitive questions at the very end. 


Questioning Approaches 

Since many retarded persons have 
both limited verbal skills and vari- 
ous speech defects,’ simply under- 
standing what they are saying can be 
a challenge, and there are undoubted- 
ly differences between interviewers 
in the extent to which they can de- 
code difficult-to-understand answers. 
Parents and others who know the cli- 
ent well may be able to understand 
much more and can translate for the 
interviewer, although as we have’ 
noted, the presence of such persons 
also has definite disadvantages 
because clients may not feel com- 


- fortable saying certain things in 


their presence. Also, parents in 
particular may take over the task 

of answering. We still prefer that 
interviews be private, and Suggest 
that the interviewer elicit repeti- 
tions of answers when they are rot 
understood. With more verbal cli- 
ents, it may be enough to ask, 
"What? I didn't hear," but with less 
verbal ones it is often necessary to 
actually reask the question ("Let me 
ask thatiagain; I couldn't get your 
answer.") Although it is frustrat- 
ing for both the interviewee and in- 
terviewer to go through this process 
repeatedly, patience may pay off. 

As long as it is done politely, most 
clients will try harder to get their 
messages through. : 


Let us illustrate the virtue of pa- 
tience. One adult living in the 
community wes asked, "What things 
do you not like about living here?" 
She said an unintelligible word 
twice; then the interviewer asked, 
“The what?", and she said the same . 
unintelligible word twice more. Af- 
ter another round, the interviewee 
finally pointed up to the sky and 
gestured to indicate that she was 
talking about noise from airplanes. 


69 


(She lived near a busy airport). 
This woman has a serious speech de- 
fect; thus there was a continual 
need to reask questions in order to 
understand what she was trying to 
say and a continual resorting on he~ 
part to gestures. It was indeed a 
frustrating interview for both par- 
ties and it did not yield much in 
the way of information, but at least 
the interviewer's patience and per- 
sistence gave the respondent the 
fullest opportunity to speak for 
herself. It is certainly preferable 
to go through this reasking process 
to obtain clarification rather than 
-‘co’jump to an interpretation of 
something said. which may be com- 
pletely wrong. 


2: 


It may be tempting to check out 

your hypotheses by saying, for ex- 
ample, "Did you say you like to go 
swimming?" Unfortunately, since we 
have discovered acquiescence to he 

a problem in interviewing retard i 
persons, this may not provide veri- 
fication of the response. Some cli- 
ents may say "yes" simply to be 
accomodating. This technique is 
potentially usable with some clients 
if you have reason to believe that 
acquiescence will not be a problem, 
but we strongly favor, reasking the 
question as the first strategy shen 
responses are not intelligible or 
relevant. In our own research we` 
generally reasked questions only 
once, but additional reaskings might 
be an even better policy. Then, if 
that fails, a good sitrategy to use 
when it is necessary to get claxifi- 
cation is to ask an either-or ques- 
tion. This often works even when 
neither choice is correct. They are 
more likely to correct your wrong 
assumption if the probe is struc- 
tured as an either-or question than 
if it has a yes-no structure. t 


In view of findings to be presented 
later on the effects of question 
phrasing and wording on the types 
of answers given; we would also em- 


phasize the need to stick with the 
interview schedule even when cli- 
ents do not appear to be able to 
respond appropriately to a question. — 
If interviewers are given the laci- 
tude to rephrase questions if the, 
original question does not yield 

an answer, they have a natural ten- 
déncy, as do adults speaking to 
children, to simplify the question, 
often by converting a difficult 
question format into an easier yes- 
no question. If the interviewer 
feels it is necessary to clarify 

a response, the question can be re- 
phrased, but the alternatives should 
be presented in the same form as in 
the original question. Sample 4.1 
illustrates this technique applied to 
a multiple choice question. As our 
data will show, this can substan- 
tially affect the responses that are 
qiven. Similarly, rewording the 
question, even without changing its 
format, may in effect change the 
question. Even a change in emphasis 
on certain words can change re- 
sponses (e.g. "Are you usually by 
yourself?" vs. "Are you usually by 
yourself?"). The first will elicit 
responses based on present feelings 
more than the second because the 
word "usually" may not be processed 
unless it is emphasized. In short, 
if alternative ways of asking the 
question are carefully planned in 
advance to minimize any potential 
for altering the intent of the orig- 
inal question, rephrasing might 
prove valuable. Generally, however, 
we favor giving exactly the same 
stimulus to each person interviewed, 
and simply treating as missing data 
cases in which an answer to the 
question could not be provided. 


. Also, when presenting multiple al- 


ternatives it is important to present 
all of them.’ Do not take a subject's 
response unless he or she has heard 
or seen all of the alternatives. Of- 
ten, subjects will try to interject 
an acquiescent response after each 
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Sample 4.1 
Rephrasing a Multiple Choice Question without 


Leading the Respondent 


The following is an example of how a multiple choice question might be re- 
phrased to obtain an appropriate response without changing the formet or 


; intent of the question. The subject here had already indicated that he 


‘watches TV. 


I: DO you naegh TV a lot, some, or not much? 

S: Huh? 

I: Do you watch TV a lot, some, or not much? 

S: Most every night / 

I: Would you say that's a lot, some, or not much? 

S: When "The Hulk" is on. Sometimes I listen to Elvis records. 

I: I need you’ to pick one of the answers here. (This was said ina 
friendly manner, es the subject would not feel intimidated.) Do 
you watch TV a lot, some, or not much? (Vocal emphasis was put on 


all three alternatives equally.) 


S: I'll say sometimes. Yeah, some. 


ar . 
' Contrast the rephrasing techniques used above with those found in the 


following example. Note. how the interviewer leads the subject into a respor-2. 


I: Do you watch TV a lot, some, or not much? 
`S: Every night I can. 
I: Is that a lot? 


S: Yeah. 
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alternative is presented. To avoid 
this, do not pause between alterna- 
tive, even if a response is inter- 
jected. You will probably have to 
repeat the queStion, but by doing 
this on the first asking, you have 
indicated that a yes-no response is 
not appropriate. Most subjects will 
wait for the full presentation on the 
second) asking. Non-verbal cues are 
also beneficial. Repeat the phrase. 


while you are looking at the inter- 
view schedule. When all alterna- 
tives have been. given, then look at 


the client to indicate you want a 
response.. 


| 


Similar’ logic applies to the issue 
of probing in response to inter- 
viewee responses tc seek additional 
information. In some interview stud- 
ies where in-depth material is 
sought, this kind of probing can be 
invaluable because it allows the in- 
terviewer to build on what the cli- 
ent says to construct a fuller pic-- 
ture. However, we feel that such 
probing techniques should alsc be 
worked out in advance. We wovld 
again warn against the use of yes-no 
questions as probes, and would favor 
either-or questions and open-ended 
questions in order to minimize the. 
chances of leading the respondent in 
a direction he or she may not have 
intended to go. Similarly it is 
dangerous to use reflzctive tech- 
niques of encouraging interviewees 
to talk more (e.g., "You didn't like 
that houseparent, did you?") unless 
you are completely confident of your 
understanding. Again, less specific 
Lypes of xeflective feedback, simply 
saying “uh huh," "oh, I see," and the 
like, are preferable. The transcript 
in Sample 4.2 illustrates an inter- 
viewer's use of probes to clarify a 
response without leading the respor- 
dent. 


There are often occasior:s in inter- 
viewing retarded persons when it be- 


comes very difficult to separate fact’ 


from fancy. Semetimes, what is being 


said simply does not sound plausible. 
For example we interviewed a man who 
claimed that he was married to the 
person assigned as his citizen advo- 
cate and no amount of probing would 
dissuade him from that story. An- 
other adult living in the community 
was asked the question, "Is anybody 
teaching you about using money. 
now?" She replied, "I did have, but 
she was working for the state but 
she charged money and I couldn't 
afford to pay." Later, when she was 
asked if she received help from the 
vocational rehabilitation office, 
she returned to this theme, but even 
-through additional probing it proved 
impossible for the interviewer to 
clearly establish who this person 
might have been, for the closest the 
interviewee came was to describe her 


: as a social worker and relate how 


: the woman did not like the client's 
barking dog but still "begged" to 
‘come every week. She finally told 
‘the woman not to come: "I said, 
'Take my money? No thank you.' -I 
said I ain't got that much money. h 
jpay everybody comes to visit me. 
‘Because clients so often do not know 
what agencies people represent or 
forget their names, it is very dif- 
i ficult to follow such a story and 
determine if there is any factual © 


i! basis to it, even though the story 


itself is otherwise rich in detail. 
Verification with someone knowledge- 
able about the interviewee would be 
useful, but this cannot always be 
done. When you have clear evidence: 
that what the client is saying is 
wrong, you can eliminate the data. 
On the other hand, one must avoid 
assuming that a client is wrong, 
just because he or she describes 
such things as exploitation or mis- 
treatment by service providers, and 
one should be sensitive to the 
feelings underlying even fabricated 
stories. 
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Sample 4.2 


Example of Probing Techniques 


In one of our studies involving unstructured interviews: with aas 
living in the community, the interviewer was free to pursue answers to 
structured questions through probes, but attempte i to do so without lead- 
ing the interviewee, as illustrated in this ae: wher s= ch probing was 
essential to establishing whom the interviewee was talki: ut. In 
response to a question asking who her best friend was, the interviewee 
said “Lois" and then setet dives Strnerores questions about the friend, 


the interviewer asked, “How did you meet Lois?" 


S: The lady I used to have wes named Donna, and she. I don't know how 
to say it. She got me to know her and that's how. 

I: Now who is Donna? How did you know her? 

S: My counselor. 

I: Was Donna your counselor, or did your counselor introduce you to 
Donna? 


She was my counselor and she introduced me to Lois. 


p 
h 


I: O.K. Where does Lois work. 

S: en 

I: ace just a friend, or does she help you? 

S: She helps. She trying to help me find a job, too. 

I: O.K. Is Lois a counselor, or is oe just a friend? 

S: A counselor 

I: Do you know who she's a counselor for? 

S: It's on Road, the cae I TRA the place on __ Road. 
(With the help ı nswer to a subsequent que... .., i: 3 £irmly es- 


tablished that Lois was a rehabilitation counselor.) i 
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Closing the Interview 


After the official interview is 
completed, we ask if the- respondent 
(1) has anything to add or wants to 
talk about anything else, or (2) has 
any questions he or she would like 
to ask. It seems only fair, since 
they have helped us, to give them a 
chance to use our time as hey wish. 
Sometimes, if interviews }Ł ve delved 
into sensitive or bothersome areas, 
tne client may want to continue talk- 
ing about these areas and this may be 
an important opportunity to work out 
any unresolved tensions. If you 
know that a client became upset about 
some topic raised in the interview, 
it may be helpful to allude back to 
it afterwards and give the inter- 
viewee a chance to let off more 
steam. This is also often the time 


-to let the client show you something 


or introduce you to someone. 


In our experience, informal chatting 
at the end of the interview often re- 
veals more insights than the inter- 
view itself. Interviewees are more 
relaxed because what: they. say is "off 
the record." In some cases, they may 
express negative {=i ons which they 
were afraid to ex, sə while they 
were "on the record." It is, of 


course, unethical to surreptitiously 


tape such comments, but you may ask 
to leave your *ape recorder on if 
this information is to be collected. 
Writing dovm what they say puts the 
situation "on the record" again and 
discourage’; openness. Another possi- 
bility is‘to take notes after you 
have left, particularly if what is 
said adds a different perspective to 
what was gained in the official in- 
terview. We are not sure what. to do 
with such information when it clearly 
contradicts responses given in the 
interview. In a small-scale study, 
one might actual _cvise answers in 
the interview v1: sis of infor- 
mation surfacing in post-interview 
discussion, but there are dengers in 
doirg so. It may be best to leave 


t 


answers as they were but draw on 
post-interview comments to enrich 
one's understanding of the perspec- 
tives of retarded persons or prepare 
in-depth case studies. 


Our main point is that you as the 
interviewer owe the interviewee 

some cf your time If he or she 

wants it. (Sometimes they do not, 
and the task is one of recognizing 
that even if they are too polite to 
say so.) It is rude and possibly 
disturbing to the interviewee to 
make a hasty exit just as soon as 
you have gotten what you wanted out 
of the relationship. We prefer 
viewing the interviewer-interviewee 
relationship as a human relationship 
and would like to think that the in- 
terviewer leaves in such a manner 
that he or she would readily be wel- 
comed back. Profuse thanks to every- 
one involved for their time and help: 
are obviously. called for. 


Recording Interview Data 

In our research, interviewers 
have attempted to write verbatim 
responses as ‘ov are given. Al- 
though tape re ‘ing of interviews 
can be useful, it .. ~ two major dis- 
advantages. First, it can be a dis- 
traction or source of anxiety for 
some subjects and may make them less 
likely to be fully honest. This may 
be alleviated somewhat by explaining 
that the tapes will not be heard by 
anyone else, so that you're taping 
the. interview so you can listen to 
it yourself to make sure you got 
everything that was said. Second, 
transcription of tapes is extremely 
time-consuming and difficult, es- 
pecially when the quality is poor due 


: to background noise in residential 


settings or «on tho interviewees 
have speec.: “ficulties or impedi- 
ments. in studies involving in-depth 
and largely unstructured interviews, 
there is probably no alternative to 
taping, but we would still recommend 
that the interviewer take as many de- 
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tailed notes on the spot as possible 
and use the notes as the primary 
source of data. Understanding at the 
time of the interview is typically 
greater than understanding days later 
when listening to a low quality tape, 
partly because mouth movements and 
body lanauage are aids to understand- 
ing. In short, we would recommend. 
use of tapes as back-ups but not as 
primary data sources. We ourselves . 
used tape in this fashion in conduct- 
ing relatively unstructured inter- 
views with highly: verbal adults liv- 
ing in the community. 


Otherwise, we have found it feasible 
to write down verbatim responses as 
most retarded persons are not highly 
verbal. £ they do launch into ir- 
relevant monologues, these responses 
do not need to be recorded, Writing 
does interfere to an extent with 
making eye contact and maintaining 
rapport, so it is important to bear 
that in mind and attempt to maintain 
eye contact as much as possible. Oc- 
casionally we have encountered an- 
other unusual problem. A few clients 
will stop talking when they see that 
the interviewer has stopped writing 
or has run out of room on the inter- 
view schedule to write a response. 
This is a minor problem that can be 
prevented by leaving large amounts of 
space after questions likely to eli- 
cit long answers. Nonetheless, it 
illustrates the fact that a host of 
unanticipated variables affect be- 
havior in an interview situation and 
the more that can be aneee a pares and 
controlled the better. 


Finally, we recommend that the inter- 
viewer go over the written notes im- 
mediately after the interview to make 
sure that nothing important was in- 
correctly or uninterpretably written 
down. . This is essential singe memory 
fades rapidly. 


Coding and Preparation for Analysis 
To the extent possible, cate- 
gories of response should be planned 


in advance of data collection so 
that interviewers can simply mark 
response codes during the interview 
rather than constructing them on the 
basis of written accounts of re- 
sponses. This is quite feasible if 
the interview is highly structured 
and uses yes-no, true-false, either- 
or, and multiple choice formats. It - 
is also feasible with many open- 
ended questions in areas where sone? 
previous research has been done and 
one can anticipate what the likely 
response categories will be or has a 
clear idea of how a universe of re- 
sponses should most logically be 
carved up. Typically, however, 
there are some open-ended questions 
for which categories of response 
cannot easily be preplanned. When 
in doubt, it may even be preferable 
to wait until the responses are col- 
lected rather than to use preplanned 
codes and later find them to be in- 
appropriate. On the other hand, our 
experiences in developing coding 
categories for open-ended questions 
given by mentally retarded persons 
have been frustrating. As noted in 
Chapter 5, such responses are di- 
verse and often strange, and the 
category schemes evolved are 
sometimes less. than aesthetically 
pleasing. 


In forming categories of response, `, 
one aims for a delicate balance 
between the general and the speci- 
fic. We have typically used all 
responses to formulate categories, 
but in a large study a sample of 
responses would suffice. In any- 
thing, it is better to err on the 
side of too much specificity. Cate- 
gories can always be collapsed later 
if only general categories are of 
interest, but general categories 
cannot as easily be broken down 
again into. more specific ones. In- 
evitably studies of this nature 
involve some recoding after cate- 
gories have initially been devel- 
oped. 
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Intercoder reliability may be a 
problem with some questions. We 
have usually established such reli- 
ability informally by havina inter- 
viewers, after they have atempted 
to establish ‘a set of coding cate- 


_gories based on similarities of re- 





sponse, explicitly draw up common 
conventions they will use in decid- 
ing which code to apply. We have 
also done small-scale intercoder 
reliability checks on two or three 
interviews to insure that two dif- 
ferent coders do indeed code the 
same responses the same way. Much 
depends on the specificity of the 
coding categories. It is straight- 
forward to code a response to a sim- 
ple factual question as "yes" or 
"no," but open-ended questions often 
leave room for subjective interpre- 
tation and some type of reliability 
check should be conducted. 


To facilitate preparation of inter- 
view data for computer analysis, two 
approaches seem particularly attrac- 
tive. In our own. research, we used 
interview schedules which included 
in the right-hand column variables 
and computer column numbers (see 


example in Table 4.1). The interviewer 
can either fill in the code on the 
spot, or, as was the case in 

search, due to the fact that 

of both responsiveness and conte 

of response were done, fill it in 
later. Several blank columns can be 
categories for open-ended questions 
have been developed. The advantage ` 
is that a keypuncher can punch di- 
rectly from the interview schedules, 
and the step of transferring re- 
sponses to computer coding sheets 
can be bypassed. ‘ - 


Alternatively, new technologies in- 
volving optical scanning can be used 
to bypass keypunching and the inevi- 
table errors it entails. ` Response ` 
codes can be entered on specially 
printed optical scanner sheets, 
entered into the computer, and then 
analyzed in any manner in whiċh data 
entered by card can be analyzed. 
Such sheets can be printed to the 
researcher's specifications. This 
approach is especially useful with 
questionnaire data or tests because 
respondents can supply their own 


_ responses directly on the scanner 





Table 4.1: Coding System Used On Interview Schedules 












l. &re you usually happy or sad? 


Sad = 1 


2. Are you usually with other people? 


No = 1 


(Olora ) 


Happy = 2 





Yes =.2 


(wR haua) 


` 





The blanks in the right-hand column permit keypunching directly from the actual interview 


schedules. 


The respondent answered the first question "sad." 
column indicates content coding, and the "sad" response is accordingly coded 1. 
indicates a responsiveness code; the 8 represents a minimally appropriate response. 


The "C" in the right-hand 
The "R" 
The numbers 


beneath these coding blanks indicate the correct column on the cards used in keypurching. 
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sheets. It would not be feasible to 
print an entire interview -schedule 

cn uch sS. -, but this approach 
might reduce the number of errors 
made in translating data into analyz- 
able form, for the interviewer could 
complete the sheets directly from 

the interview protocol. 


Based on one experience, we would 
recommend against the use of what 


‘is called free format coding in 


which the keypuncher works directly 
from interview schedules and uses 

a special notation system which does 
not involve assigning data to card 
columns during punching but rather 
doing so later. While this tech- 
nique may make it easy to transfer 
data from interview schedules to 
computer cards, it allows room for 


_ Many keypunching errors and involves 


a number of technical problems in 
then converting the free format data 
to an analyzable form. Because it 
demands more technical sophistication 
than other approaches and because we 
once experienced numerous problems 
with it, we do not recommend it. 


Conclusion 
In this chapter, we have at- 
tempted to share some lessons we have 


~ retarded persons. 
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learned about the process of inter- 
viewing retarded persons. Unlike 
subsequent ch nters, this chapter is 
based on subjecvcive impressi 

rather than hard data. As we have 
suggested, most of the guidelines 
for interviewing members of the 
general population apply equally 
well to interviewing retarded 
persons. We have attempted to 
supplement those guidelines to 
prepare interviewers for some of 

the special problems they may 
encounter in interacting with 

While we have 
tended ~to emphasize problems, we 
should point ut that the majority 
of persons we have interviewed have 
been eager to participate; tremely 
cooperative, and very gratifie 
the experience of having someone 










` seriously listen to what they have 


to say. Similarly, most parents and 
houseparents have been cooperative, 
often because they hope that research 
findings will result in improvements 
in the service delivery system. 
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Chapter 5 


TO WHAT EXTENT CAN MENT LLY RETARDED 
PERSONS RESPOND TO INTERVIEW QUws.sc ©? 


An interview is not a very fruit- 
ful information-gathering technique 
if interviewees cannot answer ques- 
tions. In view of the limited verbal 
skills, of mentally retarded persons, 
we decided that the first challenge 
in interviewing the’ retarded is to 
obtain a response of some kind. Thus 
we measured and. analyzed the respon- 
siveness of interviewees, their 
ability to meet the demands of a 
particular question with an answer 
suited to the form of the question 
(e.g., "yes" or "no" to a yes-no 
question) and relevant to the content 
of the question (é.g., about work if 
the question is, about work). We, were 
guided by the following questions: 


1. What can be expected of men- 
tally retarded persons in ` 
interviews, and what types of 
respondent behaviors ‘consti- 
tute problems in interviewing 
the retarded? 


2. To what. extent is responsive- 
ness to an interview a predict- 
able individual behavior, and 


what factors are related to it? 


_ of the question, 
cated in Chapter 3, we also measured 


2 , 
3. How can responsiveness be 
optimized? 


To convey what we have discovered 
about responsiveness, we will take up 
the following topics in turn, drawing 
from the various studies to discuss 


each topic: specific types of 


responses, the stability of respon- 
siveness, the relationship of respon- 
siveness to IQ and other client 
characteristics, the relationship 

of responsiveness to question type, 
and the optimization of responsive- 
ness. 7 ' 


Specific Types of Responses 

In most of our analysis, we dealt 
with-a measure „of the percentage of 
questions asked.of a respondent which i 
were answered at least minimally 
appropriately; that is, which met- 
the formal and substantive čemands ~ 
However, às indi- 


various types of inappropriate or l 
uncodable responses, and were inter- ` 
ested in’ the extent to which each 
might pose problems.in an interview. 
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Table 5.1 presents the mean percent- 
“ages of responses which fell in 


‘each of the nine responsiveness codes - 


in the two institutional samples. 

In each ca:.: they are based on 
responses tv one of the two paral ‘el 
interview schedules used in these 
studies {82 questions for the chil- 
dren's study and 142 questions for 
the adult study). \ Profour. ily 
recarded adults were excluded beca se 
the vast majority of them failec 

the screening interview and we: 
thereby defined as totally unres,... 


sive. Their exclusion makes the 
child and adult samples comparable 

in IQ (M=42.08 for children, 39.76 
for adults). The means in Table 5.1 
cannot always be taken as representa- 
tive of typical performance because 
some distributions were unusual (for 
example, some clients never gave a 
response while some others never 
failed to respond in some way)’. ` 
However, they are suggestive of the 
freque:.cy with which various types 

of inte~vziew behavior can be expected 

» occur. i 


. 


EE 
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Table 5.1: Mean Percents of Responses Falling in Each Category 
| in Institution Samples 


; $ i LSS 


' „Responsiveness Code 


l. No response 
2. Unintelligible 
3. Irrelevant 


4. Don't know 





5. Inadequate 


6. Request for more information 





7, Refusal to answer 
8. Minimally appropriate 


9. ‘Expanded appropriate 
m 








` 


Adult 


Child 
Institution Institution 
(N=42) * (N=52) 
6.55 
4.71 4.91 
3.48 5.11 
.28 2.85 ; 
3.73 7.41 
1.01 3.32 
ol 333 
63.63 57.94 
12.99: 11.58 
100 .00% 


100.00% 


a 


*Excludes those with IQs belcw 20, who ware generally totally unresponsive, to make 


groups more comparable in IQ. 


We can interpret these figures either 
of two ways. We can Say, 
that the typical child interviewee 
was able to give a minimally appro- 
priate amswer to 63. 63% of the ques- 
tions on the interview schedule. 
Alternatively, we can say that of 

all the béhaviors elicited from the 
sample by interviewers" questions, 
63.63% of them were minimally appro-' 
priate answers. Clearly in both 
samples we obtained minimally appro- 
priate or expanded answers in a 
majority of cases, more often minimal 
than elaborated. 


What of the inappropriate types of 
respondent behavior? Generally, a 
high percentage of inappropriate 
responses took the form of no 
response at all. Unintelligible 
responses were also a problem, 
unintelligibility, of course, being 
a function, of the interviewer's or 
listener's capabilities as well as 
of the interviewee's communication 
skills. ‘It is highly likely that a 
person who knew the respondents might, 
well have been able to decipher many 
responses that our interviewers had 
‘td judge unintelligible, but in most 
surveys interviewer and interviewee 
are not familiar with one another. 
Irrelevant responses, that is, 
answers that appeared unrelated to 
the substantive demands of the ques- 


tion, were a frequent problem as well, 


often because interviewees did not 
understand a particular question. but 
in some cases because interviewees 
had a general lack of comprehension 
cf the interview but chose o speak. 
Somewhat more common still were 
inadequate responses, which were 
judged to be relevant to the question 
but too vague or ambivalent to be |, 
coded into content categories. At 
times respondents indicated that 
they did not know the answer, but 
this was fairly rare. Our coding 
system was based on an assumption, 
usually but not always v-lid, that 
respondents should have known how. 


A 
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for example, 


to answer the questions we asked. 
Similarly, interviewees occasionally ` 
requested more information about 

the question before answering, 

usually by saying "huh?" to elicit 

a repetition of the qiusticn, but 
this did ‘not happen often either. 

By far the ‘rarest event was g ‘refusal 
to answer. the question, even though 
participants had been told at the 


‘Start that they did not have to 


answer any question they did not 


' want to answer. 


The relative frequencies of the nine 


types of responsiveness codes were 


- much the same. in the two institu- 


tional samples as well as in.other 
samples. We would have to conclude 
that interviewers of the retarded 
can expect to encourter a wide range 
of inappropriate responses, but that 
appropriate responses can generally 
be expected a majority of the time. 


The. Stability 


- of Responsiveness Scores — 


In both of the ‘institution samples, 
similar interview schedules were 
administered a week apart, allowing 
us to examine thé extent to which 
responsiveness is a consistent indi- 
vidual behavior. ‘In the children's 
institution sample, Form A consisted 
of 82 questions and Form B of 65 
questions, with 31 questions repeated 


-in exactly the same mann2r on both 


administrations and the remainder of 
the questions typically representing 
alternative ways of seeking the same 
information. A total. responsiveness 
score on each form was calculated 
for each subject (the number of 
questions answered appropriately 
divided by number of questions asked). 
When responsiveness to Form A and 
responsiveness to Form B were cor- 
related, the result was a highly 
significant correlation of .96,>indi- 
cating that the extent to which an 
interviewee was responsive to one 
form could be predicted almost per- 
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fectly from his or her responsive-’ 
ness to the other form. Although the 
correlations were lower, the frequens 
cies of specific types of response on 
Form A and B were also significantly 
related, with the exception of the 
category "Request for More Informa-, 
‘tion." These correlations were .54_ 
for no response, .74 for unintelli- 
gible, .39 for irrelevant, .30. for 
don't know, .33 for inadequate, .20 
for request for more information, 

.92 for minimally appropriate, and 
.54 for expanded appropriate re- 
sponses. Thus, the extent to which 
various less-than-appropriate and 
appropriate responses occurred was 
somewhat predictable from time to 
time. 


In the adult institution sample, 

this finding of stability was con- 
firmed. Indeed, the correlation be- 
tween total responsiveness scores 
for the two forms (Form A, 142 items, 
Form B, 131 items) was identical 
(.96). 


As another way’ of examining the sta- 
bility of responsiveness scores, we 
analyzed the extent to which the 
group's level of responsiveness 





a p 
7 was 
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. changed from first to” sécond 


administration, -In ‘the children's in- 
stitution sample, this analysis focuses 
on 3l-questions repeated exactly the 
same on both-administrations. A Latin 
square design was used to exam.: 2 the 
administration effect in the.,coitext 
of interviewer effects (see ‘Table 5.2). 
Overall, there was not a significant 
difference between first and second 
administration responsiveness, 
although scores did increase somewhat 
from first to second administration 
(from 71.8% to 74.6% overall). There 
was also no overall interviewer 
effect, that is, neither interviewer 
elicited. higher rates of responsive- 
ness from the sample. There was 
however, an’ interaction between 
interviewer and administration, 

F (l, 49)/= 9.92, p = .005. Inter- 
viewer Two, for some reason, was 
associated with a greater increase 

in responsiveness from first to 
second administration than was Inver- 
viewer One. It is possible that the 
26 subjects whom she interviewed on 
second administration were, for 
reasons we cannot explain, more sus- 
ceptibls to practice effects than 
were the 26 subjects who had Inter- 





Table 5.2: 
í * and Interviewer* 


Responsiveness of Children As a Function of Administration 





Administration One 


Administration Two 





| 


Interviewer 






sFor an N of 52 in severe +o mild ranges. One half of the sibjects had Interviewer 


One first and half had Interviewer Two first, and then Bie ather Mitarvicier cones? 


the second administration. 
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viewer One on second ađministration. 
or it is possible that Interviewer 
Two became more relaxed, warm, or š 
stimulating in manner in the second 
round of. interviews. 


A similar analysis for the adult 
institution sample (see Table 5.3) | 
revealed no significant differences 
in responsiveness associated with 


. administration, interviewer, or the 


interaction of the two. Again, how- 
ever, responsiveness did increase 
slightJy from first to second adminis- 
tration, and this effect was more 
noticeable for Interviewer Two. 


. kr 
To put these lines of evidence to- 
gether, then, we would conclude that 
responsiveņess to interview questions 
is a stable individual behavior. 
Although some respondents may increase 
in’their ability to respond to ques- 
tions.as a result of practice in a 
prior interview, dramatic changes 
are not to be expected. Moreover, 
an individual.'s ranking in the group 
as more or less responsive appears 
to be highly consistent from inter- 
view to interview. 


Responsiveness and IQ ` 

It is avite clear from the liter- 
ature on s..: age development among 
the mer:a2 ze-arded (see chapter 2) 
that in the i er ranges of the IQ 
continuum, more delay in language 
and speech is to be expected, and 
more totally nonverbal individuals 
are encountered. A critical question 
in our study of the feasibility of 
interviewing mentally retarded per- 
sons centered on clarifying what can 
be expected at different levels of 
“setardation. “At what point in the 
IQ continuum does obtaining answers 
to questions become infeasible? 


We set out with low expectations 
for persons in the profound range 


of retardation, but nonetheless 


attempted to interview persons with 
IQs below 20 in the ‘study of insti- 
tutionaliszed adults. It was here 
that our simple screening procedure, 
described in Chapter 4, was heavily 
used to avoid the unpleasant exper- 
ience of holding an interview when 
it was apparent’ that the subject 
could not participate. As noted, 
the rule was to accept any verbal 
or nonverbal response, whether intel- 





Table 5.3: 
and Interviewer* 


manea e 





Administration One 





Administration Two 


LEE 


Responsiveness of Adults as a Function of Administration 





Interviewer 








63.37% 
68.23% 





| 67.82% 


a 





*For 42 Interviewees In severe to mild range. One half of the subjects had 
Interviewer One first and half had interviewer Two first, and then the other 


Interviewer conducted the second administration. 
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ligible or not, as sufficient grounds 
for attempting the full interview. 


_Thus, the screening criteria were 


designed to give anyone.who showed 
the least sign of ability an oppor- 
tunity to be interviewed. 


Of the 16 profoundly retarded adults 
who were approached, 13 or (81%) 
failed the screening and were judged 
totally incapable of participating 
in an intervidw. The responsiveness 
of the three who did pass the 
screening was |deficient. Two of 
them were abic to respond adequately 
to slightly over half of the ques- 
tions, primarily yes-no questions. 
The third subject failed to give an 
answer two-thirds of the time and 
otherwise gave a high prcportion of 
irrelevant answers. 


On the whole, then, our expectations 
about the infeasibility of inter- 
viewing profoundly retarded persons 
were borne out. While three such 
persons were able to respond to some 
extent, we are not confident that 
their answers were meaningful and 
they were the exception rather than 
the rule: Any attempt to seek infor- 
mation from such persons should rely 
on other methods of data gathering; 


_ Systematic behavioral observation 


of circumstances, activity patterns 
and expressed affect, and, where 
feasible, use of manual and other 


nonverbal communication systems. 


‘All three of the major samples in the 
study (institution children, insti- 
tution adults, and community chil- 


dren) included persons in the severe _ 


to mild ranges of retardation. 
we were able to examine in each 


Thus’, 


. Sample the relationship between IQ 
„and ability to respond to interview: 


questions for persons with IQs 
ranging from 20 to 68. ~ 


In the institution. children's sample, 
the ability to respond was quite 
clearly a function of IQ. Table 5.4 
represents the mean percentages of 


responses falling in each of the 
responsiveness codes (with the excep- 
tion of refusals to respond, which 
were so rare that they are not 
reported). It can be noted that 
instances of no response, unintel- 
ligible responses, and irrelevant 
responses decreased as IQ increased, 
while the percentage of appropriate 
responses, both minimal and expanded, 
increased. In sevéral respects, 
severely retarded subjects, as a 
group, were quite different from 
moderately, and mildly retarded ones. 
Most notably, they were much more 
likely to fail to respond and much 
less'likely to provide appropriate 
responses. Overall, the percentages 
of appropriate response to Form A 
for severely, moderately and mildly 
retarded groups were 52.9%, 89.6%, 
and 93.3% respectively, a highly 
significant IQ croup effect, F (2.49)= 
22.51, p = .000L. 


However, it is important to note that 
variability was also greater in the 
severely retarded group (SD = 38.8) 
than in the moderately retarded 

(SD = 9.1) and mildly retarded 

(SD = 3.7) groups.. The respon- 
siveness of individuals in the severe 
retardation range was, in short, 
relatively unpredictable. Four » 
severely retarded children failed, 

the screening, while the most respon- 
sive dave appropriate answers 96.3% 

of the time, and the rest spanned aS 
the range in between. By contrast, — 
Ferm A responsiveness scores in the 
moderately yetarded group ranged from 
72.2% to, 100%, and in the mildly 


+ 


“retarded group from; 83.7% to 97. 5%, 


suggesting that all children in these 
IQ ranges were able to answer most 

of the questions they were asked. 
What this suggests is that one cannot 
make firm predictions about whether 
or not interviewing is feasible with 
severely retarded persons. The only 
way to tell is to attempt an inter- 
view and see how. individuals respond. 


í 
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Table 5.4: Responsiveness Codes by IO Group: 
Mean Percents of Answers on Form A 








Severe Moderate Mild 
\ , (N=20) (N=16) (N=16) 
















No response. 0.0 
\ i 
Unintelligible i a2 
, -2 
-6 
Inadequate 4.3 
. Request ie. information 1.3 
ga es j 7 = 
| 
Minimally appropriate 71.6 
Expanded appropriate 21.7 
; 99.9 
| 
| 1 
| 
e 
For this sample the overall correla- p= .04. On Form B, the same pattern | 
tion between responsiveness of Form A emerged (severe, 56.3%, moderate 76.6%, 
and IQ was .62, and between respon- and mild 75.5 %, F = (2, 39) = 4.46, 
siveness, of Form B and IQ .76, again p= .02. In this sample, one severely 
i indicating a strong relationship B retarded person failed the initial 
? between IQ and responsiveness. At © screening. In addition there was one 
. the same time, there was a negative mildly retarded subject whose emo- 
relationship between IQ ana the: tional disturbance seriously inter- 
percentage of "nó response" codes . fered with participation in the 
(r = .59 for ‘Form A, -.52 for Form B). interviews and whose scores lowered 
as well as between IQ and the percent-| the mean for -the-mildly-retarded--- - 
age of all other inappropriate - group and incréased the extent ‘of 
responses combined (r = -36 for Form variability. ‘This woman was not able’ 
A, -7.44 for Form\B) . to respond to 90.3% of the questions 
X ene dult i tòt ti ié, much on Form A and 85.3% of those on Form. 
i ACUZE a er Munn. B. Her exclusion from the’ analysis g 


the same pattern ofi finazngs emerged.? 
Overall, of Form A, severely retarded 
persons averaged 58.8% responsiveness, 
and were less responsive than moder- 
ately (78.0%) and mildly retarded 
(73.5%) groups, F (2, 39 =°3.63, 


increased the mean of ‘Form A for the 
mildly retarded group from 73.5% to 
81.1% and decreased the standard 
deviation from 25.5% to 10.1%. As. | 
in the children's institution study, 
variability in the severe retardation 
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- ule used in this study was, 


range was high (25.4% compared to 
11.7% in the moderate range and 10.1% 
in the mild range with the outlier 
exciuded.) 


In correlational analysis, the rela- 
tionship between IQ and percentage 

of appropriate responses on Form A 
was a significant but relatively low 
.35 with the one outlier in the 
mildly retarded group included (and 
-51. with her score excluded). As 
expected, given the high relationship 
between responsiveness scores for 

the two forms, results for Form B 
were Similar (.42 with her included, 
.57 with her excluded). Again, then, 
the study of institutionalized adults 
indicated that the ability to respond 
to questions is a function of IQ, 
with responsiveness dropping off 


markedly in the severe range of 


retardation, but with severely 
retarded persons differing widely 
from each other, some being incapable 
of participating at all and others 
responding much like moderately and 
mildly retarded persons. 


Our third opportunity to examine the 
relationship between IQ and respon- 
siveness was in the children's com- 
munity sample. The interview sched- 
in our 
opinion, easier than those used 
previously because we had by then 
analyzed responses to the questions 
used in-the institution samples and 
attempted to refine techniques to 
optimize responsiveness. . In this 
sample, responsiveness was indeed 
higher. Seve.cely rétarded persons 
did relatively well, averaging 78.5% 
responsiveness, compared to 82.1% in 
the moderately retarded group’ and 
92.7% in the mildly retarded group, 

F (2,54) = 5.93, P= .005. In this 
case, both the severely and moder- 
ately retarded. groups were found to 
be significantly less responsive than 
the mildly retarded group. The over- 
all correiation betweer responsiveness 
and IQ was .41. Moreover,:sn this 


ur 


sample, the extent of variability 
among severely retarded persons was 
not higher than that among the two 
higher IQ groups (SD = 12.8 for 
severe, 18.4 for moderate, and 5.0 
for mild). In our opinion, by 
reducing the number of open-ended 
and multiple choice questions and 
otherwise working to simplify the 


types of questions that were asked, 

we may have succeeded in making the Se 
task simpler for the community sample 

and reduced the extent to which 

severely retarded persons appear to 

be deficient compared to their higher 

IQ peers. For this reason, these 

figures do not provide a basis for 
concluding that severely -retarded 

persons in the community are more 


verbal’ y skilled than those. inan 0-00. ===- 
` institution. 


However, wé were able to compare the 
three samples more directly by exam- 
ining their responsiveness to 22 ques- 
tions asked in exactly the same way 

in all three samples. These questions 
included seven verbal yes-no questions, 
nine yes-no questions about chores 
accompanied by pictures, two either-or 
questions, one multiple choice ques- 
tion, and three open-ended questions. 


able 5.5 presents the average percent- 
age of questions answered appropri- 
ately in each of the three samples), 

at each of the three different levels 
of mental retardation. As the table _ 
suggests, the čata for tiie different 
samples were somewhat different, and 
‘this was reflected in a Significant in- 
-teraction effect in a 3 x 3 analysis of 
variance between sample and level of 
rejardation, F (4, 140) = 4.60;p = 
.002. This finding required that we 
look more carefully, at IQ group differ- 
ences within each sample. In tne chil- 
dren's institution sample, tne IQ group 
effect was Siywificant F (2, '49)= 
14.48,p <-Uui, and t-tests indicated 
that severely retarded persons were 
less responsive on the average than 
both moderately and mildly retarded 
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Table 5.5: 








“Responsiveness by IQ Group for 2? Questicns 


Used in Three Samples 








Institu zion: children 


Community children 


Institution adults 


persons. 
sample, IQ again mattered, F(2, 

3.75, p = .03. Here, however t.he 
severely retarded group was signifi- 


52)= 


cantly less responsive than the mildly 


retarded group and the moderately 
retarded group was almost so (? = 
.065). In the adult institution 
group, by contrast, IQ group differ- 
ences overall were not significant, 

F (2,39) = 2.54,p = -09, but follow- 
up t-tests did indicate that severely 
retarded clients were. significantly 
less responsive than moderately 
retarded clients (t = 2.87, p = 
In this sample, primarily because of 
the almost total unresponsiveness of 


one emotionally disturbed, mildly 
~~yYetardea woman, the mildly retarded” ~~~ 


group did fot have the highest aver- 
age responsiveness. Thus IQ group 
‘differences were weakest in this. 
groun and strongest in the institu- 
tion c:.ildren's sample. 


- The one consistent conclusion to 


be reached is that severely retarded 
interviewees generally can bè, counted 
on to be less responsive than their 
higher IQ peers, but the specific 
Magnitude and locus of IQ group | 
difference appear to vary from 

sample to sample. Over all samples, 
of course, the ‘effect of IQ group 


s 


Ih the children's comaunity 
pe 


.009). ` 


FP (2, 





Level of Retardation 





was..significant, F (2, 146) = 15.55, 
“001, with severely retarded 
persons averaging 62.99% responsive- 
ness, aS compared to 72.3% for the 
moderately retarded and 77.72% for the 


mildly retarded. 


' What of the comparison between 


samples? Overall means for institu- 
tion children, community children, 
‘and institution adults were 77.45%, 
88.92%, and 79.33%, respectively, 
and differences between samples 
approached statistical significance, 
-F (2, 140) = 2.63, p = -08. However, 


. Since mean IQs for the three samples 


were different, analysis of covar- 

iance was performed to examine 

_ differences between samples with IQ 
controlled. In this analysis, the 


-samples were not significantly dif- 


ferent in their responsiveness, 
145) = 1.16, p = .32) In 
other words, the adult sample had 


` no clear advantage over samples of 


children, and, more importantly, 


- the -apparent advantage of communi- 


ty children over institutionalized 
persons vanished when IQ was con- 
trolled. Thus, contrary to some 
evidence in. the literature, institu- 
tionalized persons were not found 
to be notably deficient compared to 
communitss residents in communication 
skills required to answer questions. 
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Relationships of Responsiveness - 


to Other Personal Characteristics — 
Although we consistently found 
that responsiveness was related to 
IQ, we found no consistent evidence 
that responsiveness was a function 
of other client characteristics. It 
is most useful to focus on the 22 
questions analyzed above which were 
repeated in all three samples. 
‘Correlations between sex and percent 
of the 22 questions answered ade- 
quately in the three. samples ranged 
from -.02 to .11, suggesting no rela- 
tionship between sex and responsive- 
ness.. Similarly, correlations 
between age and responsiveness were 
weak (-.09 to .12). Of course, 
because the age range in the two 
children's samples was restricted 
to 12-16, there was not much oppor- 
tunity for a relationship to mani- 
fest itself in these groups. However, 
the correlation of -.09 in the adult 
‘ samples where ages ranged more widely 
(SD = 4.41) was not siynificant 
-@ither. As we noted above, there 
were no recognizable differences 
between institutionalized and nonin- 
stitutionalizea groups either, con- 
trary to some evidence in the liter- 
ature of language deficits among the 
institutionalized. In short, our 
search for other subject character- 
istics predictive of the ability to 
respond in an ‘interview situation 
“was-not productive. However, we did 
find relationships between respon- 
siveness and the type of question 
asked, an important set of findings 
to which we turn next. 


Responsiveness and Question Type - 
= To illustrate in some detail the 
technique used'to ana’"ze differences 
in responsiveness as a function of 
question type, we will first examine 
the relationships existing within the 
institutionalized children's sample 
and “tiren explore the extent to which 
it was’ similar in the “other oe igi ua 


OF. QUESTION. TYPE .__.. 


RESPONSIVENESS AMONG INSTITUTION 
CHILDREN AS A FUNCTION 


While most of the children' s` 


- institution interview cóħsisted of 


simple yes-nọ questions, there were — 
other types of questions used which 
permitted a comparison of severely, 
moderately, and mildly retarded chil- 
dren's ability to respond appropri- 
ately tuo questions as a function of 
question format. Six scales were . 
formed, each measuring the percentage 
of questions of a given format that 
were answered with at least a mini- 
mally appropriate response. The 
subsets of items used to calculate | 
these scores were as follows: l 


l. Factual Yes-No Questions. Ten 
questions about activities (Do 
you: ... read books, magazines, 
or newspapers; go out to the 
movies; go out to eat; go.to 
church; go to stores; watch TV; 
listen to the radic or record 
player; play any sports; go to 
school or take classes; do any 
arts and crafts like ceramics 
or painting?) 


“) 2. Subjective Yes-No Questions. 


Eight yes-no questions, consist- 
ing of four pairs on the same? 
topics, included as part of an. 
attempt to study acquiescent 
response set (Are you usually 
happy? Are you usually sad? 
Do people here treat you nice? — 
Do people here treat you mean? 
Is this place where you live 
big enough? Is this place 
where you live too little? Are 
you usually by yourself? Are 

; ‘you ‘isually’with other: people?) 


3. Either-Or Questions. Eight 
either— or questions covering 

' the same topics as in scale 
2 above; four pairs with dif- 
ferent orderings of terms~ 
(e.g., Are you usually happy 
or sad? Are you usually sad . 4 
or happy?). 
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` 4. Verbal Multiple-Choice Ques- 
l tions. Three questions, each 
_ Supplying four answers from. 
which to select (How often 
does anybody in your family 
come to see you? A lot, some- 
times, not much, or never? 
How many friends do yeu have? 
‘A lot, some, not many, or 
none?) 


5. Pictorial Muitiple-Choice 
Questions. Three questions., 
each of which asked the 
respondent to select a happy 
face, Slightly happy face, 
slightly unhappy face, or 
unhappy face as descriptive 
of his or her feelings 
nicture shows how you feel 
about living here? Which 
picture shows how you like 
the food here? Which picture 
shows how/ you like the people 
here?) kT 


6. Open-Ended Questions. Five 
open-ended items (If you had 
one wish, what would you wish 
for? Who helps you the most 
when you need help? . What are 
they teaching you there; (at 
school)? Where do you get 
your money? What do you and 
your friends usually do to- 
gether?) i 


With the exception of the yes-no 


„items which. were sampled, the rest ` 


of these scales included all items 

on the two forms of the questionnaire 
fitting each format. Our hypothesis 
was that yes-no questions and picto- 


~ rial multiple-choice questions would 


pe associated with the highest ~~ 
responsiveness rates, and that the. 
other questions with closed’ formats 
(either-or and verbal multiple-choice) — 
would elicit higher rates of response ` 
than would open-ended questions which 
require that respondents frame the 
response. a i 


.of question for each IQ group. 


(Which s 


Figure 5.1 shows the mean percentages 
of appropriate responses On each type ~ 
This 
analysis by question, subset indicated, 
first of all, that there were signif- 
icant differences among responsive- 


‘ness rates in the total sample as a 


function of question type (F (5, 45)= 
23.90, p = .0001). 
who.had subscale scores -for each-of- -- 
the six formats (two visually impaired 
persons could not complete the picto- 
rial multiple-choice section), this 
analysis of variance by question type 
was followed up with t-tests for J 
dependent measures: to . locate ` \pbecific 
differences among question types (see 
Table 5.6). Factual yes-no questions 
and pictorial multiple-choice ques- 
tions were found to be significantly 
easier to answer than all other types 
of questions and were not signifi- 
cantly different from one another. 

The subjective yes-no questions were 
significantly more difficult than 
factual yes-no questions (t (49) = 
3.41, p = .001) and bordered on being 
more difficult than pictorial multi- 
ple choice questions as well (t (49)= 
1.95, p = .057). Either-or questions . 
ranked next in difficulty, being 
associated with significantly higher 
responsiveness rates than verbal 
multiple-choice and open-ended ques- 
tions. Correlations among the six 
subscales (see Table’'5.7) indicated 
that, generally, children s levels 


question | were predictive of’ their 
responsiveness on other types. All 
of these correlations were signifi- 
cant well beyond the .05 level. 
However, correlations among simply 
structured questions were generally 
of a larger magnitude than were 
correlations involving open-ended 
questions or the. four-choice verbal 
multiple-choice questions. i 


The analysis points out the advantages 
of using yes-no sand pictorial multi- 
ple choice questions when the goal 
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Figure 5-1 Responsiveness to Various Question Types as a Function of Level 
of Retardation. -o 


‘ s 


is to obtain responses from as many 


mentally retarded interviewees as 


possible. As the difference- between ` 


factual and subjective. yes-no ques~ 
tions indicates, the content of the 


question has some bearing on the 


likelihood of obtaining a response, 


u 

but is relatively minor compared to 
the effect of question format. Sub- 
jective yes-no questions, and perhaps 
the ‘subjective either-or que“: ons 
used in this study..as well, otten 
forced respondents ‘to make a black- 
and-white choice. whore = responsive 
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Table 5.6: T-Values for Differences between Appropriateness of Response 
Scores on Six Subsets of Items (N=50) 












Pi 6.14* |6.37* 









ctorial multiple-choice 
Factual yes-no 6 .43* |6.30* 


Subjective yes-no 





5.20% |4.97* 


Either-cr 4.17% |4.15* 


Verbal multiple-choice 


_*Significant at the .05 level, 


ness fell off, it was often because easy to answer, undoubtedly because 
residents would request clarification they required only pointing and not 
of the question or give vacillating necessarily understanding, verbal 
responses which could not be coded nultivie-cietce questions were, sur- 
one way or the other. While picto- prisiogly, as difficult as open-ended 
rial multiple-choice questions were auestions. While this may have been 
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“Table 5.7: Correlations Aworg Approprietansss of Response 
Scores on Six Types of Questiors 
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*Significant at the .05 level. 


partially due to the topics and 
quantitative response options for 
the questions used in the study, 
tis quite likely that. four options 
are too many for many mentally 
retarded persons to consider at once. 
Certainly the two-choice either-or 
questions were much easier by compar- 
ison, despite the fact that they: 


it > 


were subjective and relatively abstract. 


Analysis of IQ differences in 
responsiveness to each of the six 

types of questions was conducted 
through one-way analysis of variance 
with follow-up t tests. Separate 
variance estimates were used due to 
higher variances in the. severely 
retarded group than in the moderately 
and mildly retarded groups. On all 

six sets of questions, IQ group dif- ` 
ferences, as indicated by F statictics, 
were significant keyond the .005 level. 
Moreover, follow-ud t-tests indicated 
that for all six question formats, 

the severely retarded group was less 
responsive than the, moderately and 
mildiy retarded groups and the latter 
two grcups did not differ significantly 
from each other. The only difference 
between the moderately and mildly 
retarded subjects approaching signif- 
icance was on the,either-or questions, 
where, contrary to expectation, the 
_moderately retarded were somewhat more 
responsive than the mildly retarded 

(t (28) = 1.83, p = .078). All dif- 
ferences between the severely retarded 
and the other two groups were signif- 
icant beyond the .01 level. 


Thus, the IQ group differences in 
total responsiveness for this sample, 
which pointed to deficiencies among 
the severely retarded, do not appear 
to be a function of the fact that 
most of the questions asked in the 
study were in yes-no-form. The 
severely retarded appeared to be less 
capable of giving appropriate 
responses to alltypes of questions 
asked. Of course, their low respon- 
siveness. rates on verbal multiple 


choice and open- ended questions, and 


to a lesser extent either-or questions, 
raise serious questions about whether 
those formats should be used at all. 
with severely retarded children, for- 
in each case, less than half of the- 
severely retarded group studied was 
able to respond appropriately. More- 
over, it was on these types of ques- 
tions that the difference between the 
severely retarded and the high IQ 
groups appeared to be most pronounced. 


RESPONSIVENESS AND QUESTION TYPE 
IN, OTHER SAMPLES 

Table 5.8 presents the mean respon- 
siveness percentages for the same 
categories of questions (although not 
exactly the same questions) in the 
children's institution, adult insti- 
tution, and child community samples. 
One can readily see that the ordering 
of the question formats is much the 
same in all three samples. Among 
institutionalized adults, factual 
yes-no questions about activities 


- were slightly easier than yes-no 


questions of a subjective nature, 

and both types of yes-no questions 
were significantly easier than verbal 
either-or questions, wnich in turn 


elicited higher responsiveness scores 


than either’ verbal multiple-choice 
or open-ended questions. Thus these 
findings paralleled almost exactly 
those for the institutionalized chil- 
dren, either-or questions being more 
difficult than yes-no questions but 


‘easier than the challenging open-ended 


and verbal multiple choice questions. 


The pattern of differences in the com- 
munity children's sample was somewhat 
_ different. Pictorial choice and fac- 
~ tual yes-no questions about activities 
were significantly easier than all. 
other types of questions. Subjective 
yes-no questions, while still falling 
next in order, were significantly 
more difficult than.these two formats | 
but yielded higher responsiveness 
scores than did the remaining formats. 
Verbal either-or questions, rather 
“than being easier than verbal multiple- 
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Table 5.8: Mean Percentages of Appropriate Response to various 
Types of Questions in Three Samples 






























Institution Institution Community 
Children Adults Children 
(N=50) (N=42) * (N-57) 





Question format 





Pictorial choice _ 


(not used) 
Verbal yes-no (factual) 


Verbal yes-no 
(subjective) 


a 





Verbal either-or . 





Verbal multiple-choice 


Open-ended 


“Excludes profoundly retarded groun. 


choice and open-ended questions, The use of verbal either-or questions 
proved to be as difficult. In this involves some sacrifice of respon- 
sample, responsiveness was generally siveness, but still can be expected 
high, and there was not the marked to yield appropriate answers from 
drop in responsiveness associated roughly two-thirds. of the sample. 
with verbal multiple-choice and - To our surprise, verbal’ multiple- 
open-ended questions as there had ‘choice questions were consistently 
been in the institutional samples. as difficult to answer as the more 


obviously difficult open-ended ques- 
tions. In our interviews, these _ 
questions typically required a choice 
among four; or sometimes three, al-. 
‘ternatives (e.g., "never," " not™ 
much," "sometimes," or "a lot" in 
answer .to a question about the ex- 
tent of involvement in some activity). 
both types of questions can be an- Apparéntly, either because a quanti- 
swered nonverbal ly (by pointing in tative concept was involved or be-. 
the first case, by nodding or . cause subjects were unable to process 
shaking the head in the second). sò. many options, these questions 


Still, the ordering of question -for- 
mats must be considered highly 
stable from sample to sample. One 
can generally expect to obtain 
answers from a large proportion of 

a sample of retarded persons if one 
asks picture choice and verbal yes- 
no questions. Not coincidentally, 


proved as difficult as Diger rene that- 
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required respondents to generate re- What of the differences among IQ 


sponses -indepen ‘-ntly, with benefit groups in their ability to manage 
of structure. i different question formats? In the .~ 
It is difficult to fully convey the © adult institution samples, differ- 
problems we encountered with open- ences between IQ groups were not 
ended questions without providing significant for subjective yes~no 
concrete examples. Samples 5.9 and questions, factual yes-no, questions 
5.10 provide verbatim answers given about activities, and mułtiple- 
by virtually all members of the choice questions, although trends 

= adult sample who could respond ver- pointed toward higher responsiveness 

N Bally in some manner. . (Wwe excluded as IQ increased. On both either-or 

"don't know" and "huh?" responses as questions and verbal multiple-choice 
well as a few repetitive answers.) questions severely retarded persons 
The tables illustrate how the were significantly léss responsive 
ability to answer such questions than moderately retarded persons. 
improves as IQ increases, but they (These analyses included the mildly 
also illustrate how few responses retarded subject who was generally 
to open-ended questions were inter- unresponsive and who lowered the 
pretable. Many adults misunderstood mean for mildly retarded subjects.) 
the questions. For example, several, On open-ended questions, the severe- 
instead of indicating what people ly retarded JTOUp Mes Sven: More 
did that bothered- them, named people seriously deficient, averaging 30.3% 
who botter’ them, and some seemed to i piace ae ae to 62.7% for 
think that they Should discüss moderately rdtarded persons and 63.9% 
things they did that are nice or for mildly retarded persons. Thus, 
bothersome, or simply things tha: in this sample, while the severely 
they like or dislike. Where re- retarded group invariably had the 
sporndents understood the questions, lowest mean responsiveness, their 
their answers did provide some clues MOst notable deficiency was on open- 
to how they view the social world ended ea - 
around them (e.g., note the comments Among the community children, a 
about noise as a source of irrita- somewhat similar pattern emerged. 
tion), but the insights to be gained while there were no significant 
are limited. Particularly for ques- differences on yes-no questions 
tions for which categories were con- about act‘‘7ities, picture choice 
structed after client and signifi- questions, and subjective yes-no 
cant other responses had been `~ questions, severely retarded subjects 
reviewed, it proved extremely dif- were significantly less responsive 
ficult to sort answers into meaning- than both higher IQ groups to verbal 
ful categories. Moréover, as the either-or questions. On verbal 
tables suggest, once one omits the multiple-choice questions, they 
irrelevant or inadequate (vague) were significantly less responsive 
_responses,-very few codable answers | than were mildly retarded subjects. 
are left to categorize in the first and finally,.once again, the most 
place. Again, while insightful — striking .IQ~group differences emerge 
answers sometimes emerge from more `. for open-ended questions. ‘ Here, 
verbal respcndents, the return on severely” retarded: respondents, with. 
investment, if we may borrow a >- ` a mean responsiv7ness score of 51.1%, 
phrase from business, is generally _ were significantly less responsive 
prohibitively low for open-ended than the moderately retarded (68.9%), 
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questions.. who in turn-were-less responsive 
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Sample 5.9 
“What Nice Things Do people Here Do for You? What Else?" 


ie Selected Responses of Institutionalized Adults* 


Profound 
‘It's from playing games on the Secora hop 
Yeah 
Severe 2 
Nice 
Like to Work 
No way 
Work 
Mrs. .___, she's my houseparent, M.s. __s etc. (list names) 
Three 
Mary P i E 
I don't do that. EPA talk is pacpie/ and if they go out, 
that all. And sometimes we go skating/Sometimes we dance: 
together. | 
We talk, that's all. 
-~ I used to play apaun and ride m bicycle. ~oo 
i Nice 
" Moderate $ v 4 i 
= play with me/play games/play hide 'and) seek/ farmer in the dell and 


= — 


= = l l hopscotch and play TO telephone 


‘They give jie books and they give me necklace/give me hane on weekends/ 
well, buy me ‘things/I don’ t think of no mo: re. 


~ I do myse1#,help me out. 


t 


Get me up in the: morning and feed me my breakfast/They, help me get 

on the pot and off the bathroom/They feed ne sometimes/They help 

me go to school/They'' re so nice to me/They treat me nice here. 
They let me go to the canteen when I want/Let me buy something when 

I want to/Come to cottage and drink a coke and stuff, take a bath 

and get yaady for supper/Just go swimming whenever we want/Sweep 

during P.M. l 
Let is go shopping/after movies/bowling 


Oh,: not much 


Let me buff the floor/Mop . 


Mild 


~ 


They help you out/Show you how es do things/(unintelligible) and make 
the bed 

They work here and clean up, mop and clean and, kiss. 

Give us ‘treats 

I help in the kitchen/I take my plate up 


Nice peoples. They been good, quiet. 


Usually they're so tied up. But when they have time, I usually go 


Se 


someplace with them/I made my bed this morning 


Work/Stay work/wash face, hair/shave/try hair spray 


v y x 


Help me out,'Help me out when I'm sick 


They do, too--buy me things, everything I need/They let me do things I 


want to/They—let me go places i 
Try to help me get a job/Help every people ™ . 77 = a 


They help me walk/They go to the canteen for me 2 


- They tell me when it's time to eat and go to class and sometimes we 
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just sit down and tell each other jokes 
‘Wash yon clothes/Talk to you. Ask you questions, talk about things 
you want to talk about. .—Look at things you want TT a | 
your clothes, talk to you, put things up.for you, pick things 
up for you/Remind you to do stuff 
Well, I help make beds and strip beds and dress the girls. And if 
I'm nice, I stay in the back and hotp taaces strip beds. If I 
stay back there, they don't have .to yell at me and they let me 
"` have a cup of coffee for breakfast and they let me wash dishes/ 
" When they're up in front, I'm usually washin' .and dryin' clothes, 
when I lived in — (goes on to talk about punishments). . 


`, 


*Slashes mark points at which the probe, "What else?" was provided. 
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Sample 5.10 
“What Do People Do Here That Bothers You? What Else? 
Selected Responses of Institutionalized Adults* 
Profound 7 A 
Yeah 
Ps 
It's from bothers you, bother, bother 


Severe 


‘(Fight F ES 


Did you find Ay screw? (for his glasses) ee 
Nobody bothers. me 
Roy bothers me. I buy hima coke, he's good (rambling on) 
-Kevin, George, James 
Nobody — >> i 
Alice does sometimes 
They do. I slap people/There's something they do and I hit 'em. 
Work 4 | 
calls me ____ and I don't like that name. 
Moderate 
Nothin’ | ! 
oh, some bother me. I get along with some, not all. 
i \ 
Make noise/They yell/They keep it 
I hate to be picked on/noise/that's ie 
They pass me around | 
Fuss 
I ante Like going around dalag stuff/Everything bethieee maypickih up 


trash. 
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Knock in the head/Knock down; soup when we have soup 


Gets you in trouble/and play 


“Nothing, noise don't hurt me. 


Mild 


Chews tobacco and stuff/smoking/pestering when I. get off from laundry / 


I can rest.- I'm tired when I get home from work but they disrupt 


me/stealing, I can't stand that/that's it 


Saturday 

Nothing 

Bad man 

Call names, cuss; they cuss and taik mean/ (unintelligible) /stare at 

l you/they kick you every day 

Talk a lot/when I try to sleep, they jake me up. That bothers me more 
than anything in the world. - 

I don't like real loud talk. I don't like fussin' or fightin'/ 
hittin’ peoplie/orie thing aay have on my medical folder, that 
whenever I was born something happened to my mind. They say 
they can't get BEP as fast as they want to. I really 
want to go home for good. I believe I'm old enough. 

They hit on me and they call me names and they bite me and they get 
me in trouble/nearly every morning, every afternoon and every - 
night, I have to stand this screamin' and yellin' that goes on 
every afternoon, morning, and night, and Í get this headache 
that won't go away. I called the housemother and asked her to 


ask the nurse to give me a couple of aspirins. But the only 


person in this cottage who gets aspirin is : 
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Sometimes they might ask questions I can't answer, or a few other 
‘things. I don't get bothered real eaSy-nothing really. 


*Slashes mark points at which the probe, "What Else"," was provided. 


O. 
ERIC 


re) 
ERIC 


Pal 
eq 
Q 
+» 
(i 
od 
H 
Q, 
O 
oe 
jer 
c 
uo 
G 
df 
O 
G 
[0] 
Q 
v 
Q 
m 
» 
fo 
a 
U 
H 
Q 
A 


: Yes-No 
Bither-Or (Pictures) 
Open-ended 


4 Either-Or (Verbal) 
* Multiple Choice 


Community 
Children (N = 57) 


Severe Moderate 


Level of Retardation 





Figure 5.2 Responsiveness to Various Question Types as a Function.of Level of Retardation ° 


than the mildly retarded (88.5%). 


These findings are strikingly por- 
trayed in Figure 5.2. 


The- pattern of results, then, sug- 
gests that while severely retarded 
persons can generally be expected to 
have the most difficulty responding 


to questions, their handicap is most: 
evident on the questions which can 
be considered the most cognitively 
demanding, particularly open-ended ' 

. questions which put the burden for 
generating an answer on the inter- 
viewee. These findings, collectively, 
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have clear implications for survey 
design where the goal is to optimize 
responsiveness. We will consider 
another means of optimizing respon- 
siveness next. l i 


Effects on Responsiveness 


of Asking Questions Twice 
Certainly one possible reason for 
low responsiveness is the inter- 
viewee's failure to process the ques- 
tion. Throughout our research, we 
used responsiveness to the first 
asking of the question as the major 
measure of responsiveness. However, 
we also used the strategy of asking 
the question twice if the inter- 
viewee had not been able to answer it 







60 = 


50 


40 


Percentage of Agreement 


First, Asking 


on first asking. Moreover, in the ` 
adult institutional sample, we ex- 
plicitly tested the value of repeat- 
ing the question as a way of increas- 
ing responsiveness. 


The analysis focused on a total of 131 
different questions from both forms 
which were asked of all interviewees. 
The measures compared were the per- 
centage of the sample responding with 
at -least.a minimally appropriate 
response after both first and second 
asking. These two figures were com- 
pared through t tests for correlated 
means for each of five sets of- ques- 
tions: 82 yes-no questions, 20 open- 
ended questions, 11 multiple-choice 
questions, 13 picture choice ques- 


———* 69.6 Picture Choice © 
68.5 Yes-No 


56.4 Either-Or 


ae 50.7 Multiple Choice 


43.5 Open-ended 


—_____—___—_ 


After Two Askings 





aaaea 


. Figure 5.3 Percent of the Sample Responsive to Various Question Formats After i 
One Asking and Aftér Two Askings 


5.24 


re) 
ERIC 


re) 
ERIC 


tions, and 5 either-or questions. 
Figure 5,3 shows the mean respon- 
siveness figures for each of these 
five sets at first asking and for 
first and second asking combined. 


The figure clearly indicates that 
for each category of question there. 
was at least some gain in sample f 
responsiveness figures associated 
with giving interviewees a second 
chance to answer the question. In 
fact, despite Jow numbers of ques- 
tions in some question: sets, all of 
the t tests were significant at the 
.0l level at least. in terms of 
sheer increase in responsiveness. 
the gains were most striking on 
either-or and multiple choice ques- 
tions, where the percentage of the 
sample ultimately responsive was 
approximately 11 percentage points 
higher than the percentage respon- 
sive at first asking. The gains 
for picture choice and yes-no ques- 
tions were moderate by comparison 
(3.6% and 5.4%, respectively), but 
this may very well have been due to 
the fact that responsivenezs to the 
first askings of these questions 
was already almost at its ceiling. 
(Although roughly two-thirds of the 
sample appears responsive in 
the figure, it must be noted that, 
these figures are for all 58 sub- ` 
jects, 14 of whom had failed the 
screening, interview and thus were 
defined as totally unresponsive. 
Thus, if all interviewees who actu- 
ally received the entire interview 
` had answered a question, the respon- 
siveness figure would have been 
approximately 76%.) Open-ended 
questions were difficult on first 
asking, and the gain attributable 
‘to asking the question twice (about 
eight percentage points) was not as- 
large as that for the almost equally. 
difficult verbal multiple-choice 
questions. By analyzing specific 
changes, we found that persons who 
had given an inadequate (vague) re- 
sponse the first time were most 


ett 


likely to give an appropriate re- 
sponse the second time, but that a 
variety of inappropriate respgnses 
could be converted to appropriate ones 
if the question was repeated. 


Summary and Implications 

As the first challenge in inter- 
viewing mentally retarded persons is 
tc obtain a usabie answer of some 
kind, we devoted considerable energy 
to measuring and analyzing respon- 
siveness to the communication demands 
of questions.. The nine-category 
coding system we developed proved to 
have higher interrater reliability 
and to adequately classify the range 
of possible behaviors in response to 
questions. There are, to be sure, 
problems in coding responsiveness. 
For example, the judgment that a 
response is unintelligible is clearly 


a function of interviewer sensitivity; 


"don't know" responses under some 
conditions should probably be labeled 
appropriate rather than inappropriate 


‘responses; and there are judgments 
“to be made in determining that an 
-answer is too vague to qualify as an- 


appropriate response, especially 
when, for example, subjects give 
relatively sophisticated but vacil- 
lating answers to yes-no, or either-or 
questions which ultimately cannot be 
construed as taking one side or the 
other. Still, the responsiveness 
coding system generally served its 
purpose of providing a means of : 
examining answering behavior independ- 
ent of the reliability or validity 

of answers. 


As indicated in this chapter, we 
discovered several important things 
about responsiveness. First, the 
reasons for failure to respond ap- 
propriately are diverse. In inter- 
viewing retarded persons, one can 
expect silence in response to ques~ 


‘tions’ as well as various kinds of 


unintelligible,, vague, and totally 
irrelevant responses. As indicated 
by the predominance of minimally 
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appropriate responses as compared 


“to expanded appropriate responses, 


one can also expect codable answers 
to generally be brief, in many cases 
monosyllabic. 


As our analyses clearly PE A 
responsiveness is a stable individual 
characteristic. An individual's 
ranking in the group based on the 
percentage of questions answered 
appropriately is remarkably stable 
from interview to interview. While 
some subjects appear to become more 
responsive to a second interview as 

a function of having participated 
previously in an interview, changes 
in responsiveness from administration 
to administration were slight. More- 
over, responsiveness as a behavioral 
characteristic appears to be related 
to intelligence. Certainly we found 
that verbal interviewing techniques 
are infeasible with the vast majority 
of profoundly retarded persons. In 
the severe range of retardation, 
individual differences were very 
large, and it was difficult to pre- 
dict which individuals would, be able 
to respond. This suggests that 
screening potential subjects for 
their ability to participate in an 
interview, possibly with more strin- 
gent criteria than were used in our 
studies, is especially useful in the 
severe range of retardation. Other- 
wise, one can generally expect most 
moderately and mildly retarded: per- 
sons to be able to answer most ques- 
tions asked in a simple survey. Our 
data also indicated that while 
responsiveness was a function of 

IQ, it was unrelated to the sex, 

age, or place of residence (insti- 
tution versus community) of the 
retarded person. 


Perhaps the most practically impor- 
tant finding about the nature of 
responsiveness is that it depends 
not’ only on the subject's cognitive 
abilities but on the form in which 
questions are asked. Judging from 
the consistent findings across three 


samples, we can confidently conclude 
that yes-no questions and quest iqns 
calling for a choice between pic~ 
tures are the question formats likely 
to optimize responsiveness. Verbal 
either-or questions also. appear to 
qualify as an attractive option if 
obtaining codable answers from a major 
ity of a sample is the goal. However, 
verbal questions requiring choice 
among three or four alternatives and 
open-ended questions appeared to be 


‘difficult to answer, particularly 


for the severely retarded. As we 

will point out in later chapters, 
there were other problems associated 
with these formats besides low 
responsiveness, and altogether they 
prove to be relatively poor sources _ 
of information in most cases. >At 

the very least, it must be recognized 
that many persons in a sample will 
simply not be heard from when these 
formats are used and that their 
utility is greatest when interviewees 
have the high verbal ability required 
to comprehend them and generate. 
answers. In keeping with these 
findings, one possible approach to 
survey design would be to construct 
layered interview schedules so that 
each area .of questioning begins with 
the relatively easy~to~answer struc 
tured. questions, especially those 
which can be answered nonverbally, 
and then progresses, where the inter- 
viewee's communication skills permit, 
to less structured questions which 
can then provide more specific 
information to clarify clLosed-ended 
responses. 


Finally; our data suggest another 
relatively simple way of increasing 
responsiveness; that is, repeating 
questions, possibly even more than 
one time if necessary, in order to 
give retarded persons additional 
opportunities to understand them. 
The simple technique of repeating ' 
a question if the first response was 
less than appropriate did increase 
responsiveness for all types of 
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questions examined. If simple repe- 
tition works, what about the common 
practice of rephrasing questions to 
make them more understandable? .We 
consciously avoided this practice 

in our research because we did not 
want to run whe risk of altering 
responses by altering questions. 

As our later findings will illus- 
trate, the way in which a question. 
is worded or formatted can make a 
substantial difference in the 
answers that are given. Unpremedi- 
tated rephrasing of questions is 
particularly dangerous, for people 
have a natural tendency to simplify 
the form of the question; for 
example, dropping back to what 
appears to be a simpler and more 
structured format if an open-ended 
format does not work. Thus, when 
an interviewee fails to answer 
appropriately the first time, we 
would recommend starting with a 
verbatim repetition of the question. 
If that does not work, responsive- 
ness might still be increased 
through the use of preplanned 
rephrasing of questions. However, 
we would recommend that these re- 
phrasings involve changes in wording 
rather than changes in format simply 
because changes in format, while 
they may be more likely to increase 
responsiveness, are also moré likely 
to alter the content of responses, 


judging from evidence to be pre- 


sented later. 


Finally, we might add a word about 
the implications of our responsive- 
ness data for the communication 
development ‘of mentally retarded 
persons. The predominant trend in 


the previous literature on the com- 


munication skills of the retarded 
has been to study the forms of lan- 
guage and the development of the 
ability to comprehend and produce 
those forms. Recently the emphasis 
in the study of normal language 
development has shifted so that 
researchers are more concerned with 


the use of linguistic forms in. var- 
ious situations, with the functions 
that language serves in communication 
settings. The measure of responsive- 


ness used here is a measure of behav- 


ior in a real communication setting; 
more precisely, a measure of an in- 
terviewee's tendency to meet the 
communication demands of interview 
questions. Although ours is not a 
developmental study, it suggests that 
there is a developmental sequence 
that children must progress through 
in understanding and answering ques- 
tions. Like some previous research 
with normal children (e.g., Ervin- 
Tripp, 1970) it suggests that struc- 
tured questions may be mastered _ 
earlier than unstructured questions 
and “Suggests more specifically which’ 
kinds of questions can be answered 
most easily by persons with low 
mental ages. We do not yet know the 
extent to which responsiveness to 
questions of increasing difficulty | 
can be. trained. .However, we are, in’ 
the context of a related study, | 
testing the effectiveness of such 
training. It would. appear to us 
that since answering questions is a 
common form of verbal eee 
explicit training in responsiveness 
should have an important place in 
language arts curricula for the re- 
tarded. Such training might first 
aim to establish minimally appro- . 
priate responses and then encourage 


| students to go beyond the minimal 


demands of the question to qualify 
and elaborate answers and to antic- 
ipate upcoming questions. Other 
research we have conducted (e.g., 
Elias, Sigelman, & Danker~-Brown, 
(1980) suggests that the ability of 
retarded adults to answer questions 
appropriately is associated with 
making positive impressions on 
raters of videotaped interviews. 
Thus, there is some evidence that 
training to increase responsiveness 
to questions might not only con- 
tribute to communication deve ToPiOne 
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but facilitate-social and vocational 
acceptance of retarded persons. 


To this point in the narrative, we 
have considered only the ability tc 
respond to questions. . However, ob- 
taining an answer appropriate to the 
questions is only part of the battle. 
One might ask whether the answers 


obtained are reliable and valid if 
one is to have confidence using 
interviewing techniques as a source 

. of information about the circum- 
stances, needs, and attitudes of 
retarded persons. In the next 
chapter, we confront the reliability 
issue. 


Q 
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TO WHAT EXTENT AR 


Assuming that an answer is 


obtaine’ from a mentally retarded 


interviewee, one must immediately 

be concerned with the reliability 
and validity of response. This 
chapter concerns reliability of 
response or the extent to which 
answers can be considered stable 
indications of interviewees' needs, - 
circumstances, and attitudes. There 
are at least two ways in which the 
reliability issue can be approached 
in survey research. One approach, 

a variation on test-retest relibil- 
ity, involves readministering the 


same questions after a brief interval - 


of time and determining the extent 

to which answers given on the two 
occasions are consistent. The second 
approach is akin to measuring test 
reliability by ene the 
consistency of performance on alter- 
nate forms of a test. It would 
involve asking questions in alter- 
native ways and determining the 
extent of agreement between responses 
to the alternative questions. In 

the present study, we did indeed 

ask a number of questions in alter- 


p. 
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Chapter 6 


E ANSWERS RELIABLE 
OVER TIME? 


e 


nate forms, but we have chosen to 
discuss the results in the context 


‘of validity rather than reliability. 


Perhaps this decision could be 
debated, but in any event, the present 
chapter's discussion of reliability 

of response is concerned only with 

the extent to which answers given 

one week agree with answers given by 
the same respondents the next week. 


Two interviews were administered 
approximately a week apart to both 
institutionalized children and insti- 
tutionalized adults. In each sample, 
half of the subjects had Interviewer 
1 first and Interviewer 2 secorid, 
while the other half had Interviewer 
2 first and Interviewer 1 second. 
Since two forms of the interview 
schedules were used {and were also 
counterbalanced for order of presen- 


‘ tation) ,. interviewees were exposed 


to somewhat different interview sched- 
ules on the two occasions. Within | 
those schedules, however, some ques- 
tions remained in identical form on 
both occasions, and the only differ- 
ence was the context of other 
questions. l 
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There is no absolute criterion which 
can be. used to make the judgment 
that responses are reliable or unre- 
liable. As we shall point out, the 
analysis of such information is very 


` complex, and multiple guidelines for 


interpreting levels of consistency 
may be called for. One can, in 
looking at the literature on test 
reliability, generally point to a 
criterion of reliability of .80 to 
.85 or higher, but this criterion is 
not typically applied to the reiia- 
bility of individual test items so 

it serves only as a rough standard. 
We will indeed examine the percentage 


of paired responses that are consis- 


ent rather than discrepant. As we 
will indicate, another perspective 
can be obtained by examining tle 
extent to which the degree o“ ¢onsist- 
ency obtained deviates from the 
degree of consistency which could be 
expected on the basis of chance; but 
this criterion alone is not entirely- 
suitable either. To complicate 
matters fucther,, one must consider 
whether the data obtained through, 

a survey are to'be analyzed at the 
aggregate or. the individual level. 

It is entirely possible that many 
individuals might change their 


_xesponses from’ one. occasion to the 
‘next but that the overall picture of 
‘the group obtained will not change 


a great deal, assuming that changes 
by individuals essentially cancel 
each other out. In this same situa- 
tion, however, it would be unwise to 
use the data to identify individual 
BOrre lates of response or to make 
program decisions affecting individ- 
ual clients. 


With that in mind, let us look in 
some /detail at the extent to which 
answers given by institutionalized 
chi/ldren were consistent from one 
week to the next. 


“| 


‘the floor. 


Reliability Over Time— 


Institutionalized Children - 

In the children's institution 
‘sample, 31 questions repeated on both 
forms were examined for reliability. ` 
“Thirteen of these were verbal yes-no 
questions about activities, quality 
of life, and rules and decision- 
making. An additional four questions 
of the yes-no variety were accompa- 
nied by pictures illustrating the 
chores aske about. fable 6.1 pré- 
sents a variety of information about 
these 17 questions and can serve to 
illustrate the issues in analysis of 
reliability. 


Generally a relatively high propor- 
tion of the sample was able to 
respond to both askings of each of 
these yes~ no questions, although. 
responsiveness fell off somewhat on 
the rules and decision-making ques- 
tions, presumably due to the’ diffi- 
culty of such concepts as "decide" 
and "allowed." Thus, the figures 
are based on a fairly wide range of 
subjects (more so than is the case 


.for open-ended questions and other 


more difficult formats). For the 
entire set of 17 quéstions, an aver- 
age of 86,9% of the respondents were: 
able to respond consistently "yes" or 
"no" to both askings. The percentages 
of consistent response, as shown in 
the fourth column: of the table, 

ranged from a low of 56.1% on the 
question about whether residents are 
allowed to hit people to a high of 
100.00% on the question about cleaning 
Generally, the percentage | 
of subjects responding to both ques- 
tions who were consistent in their 
responses were lowest questions to 
which a "yes" answer might be con- 


_strued as a socially undesirable 


response. The questions in which a 
"yes" response would indicate that 
residents are allowed to hit people, 
that, people yell or say mean things, 
and that people take things away all 
had low percentages of consistency. 





Table 6.1: Reliability of Response of Yes-No Questions: Institution Children 


ON $ Yes | % Yes | 
Responsive (Form A) | (Form B) | % Consistent | Kappa 

















Question 




















Do you watch TV? 

Do you listen to the radio or record player? 
Do you go to school or take classes? 

Do you have a family? 

Is this place your. home? 

Do people here yell at you or say mean things? 
Do people here take things away from you? 

Do people here help you when you want help? 
Do people here teach you things you. want to learn? 
“Are you allowed to stay up late at night? 

Are you allowed to hit people? — 

Do you decide what chores you do? 

Do you decide how ty spend your money? 











(With Pictures) 
Do you set the table? 
Do you do dishes? 44 90.9 
Do you do laundry? l 
Do you clean the floor? 

















* significant at the .05 level (one tailed) ** significant at the .01 level. 
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. Consistency for the question about 


decision making regarding chores was 
also low, possibly in this case 
because two difficult concepts, 
"decide" and "chore," may have cre- 
ated confusion. Generally, consist- 
ency figures for the 13 verbal ques- 
tions (average consistency 86.0%) 
were similar to those for the ques-. 
tions accompanied by pictures aver- 
age consistency, 89.8%). Generally 
the predominant form of consistency 
on most of these questions (with 


the exception of. the negative items 


discussed above) was saying "yes""^ 
both times rather than saying/ "no" 
both times. "Yes" answers predomi-. 
nated on most of these questions. 
Moreover, the percentage of subjects 
saying "yes" on Form A (r = .88) 

as well as on Form B (x = .77). 

This suggests that reliability is 
artificially inflated when marginal 
distribution is lopsided. As will 
be noted later, acquiescence was 
common in our samples; thus we sus- 
pect that the reliability figures for 
yes-no questions are generally 
inflated. We would at least warn 
that the relatively high consistency 
figures in Table 6-1 are not neces- 
sarily evidence of validity. However, 
we cannot estimate precisely the 


-extent to which acquiescence might 


have contributed to consistency. 


In any case, the figures are encour- 
aging in the sense that we can expect 
much the|same response both times 
from most subjects most of the time. 
As the second and third columns of 
the table indicate, this means that 
the pictures of the group obtained 
from two,askings of the same ques- 
tions are highly similar, the biggest 
discrepancies appearing on the ques- 


‘tion about being allowed to hit peo- 


ple and on the picture questions 
about dishes and laundry, both of 
which were associated with only mod- 
erately high consistency figures. 


A different perspective on relia- 
bility emerges from the last column 


of the table, which reports the 

kappa statistics calculated from 
each 2 x 2 contingency table analyzed. 
Kappa is a straightforward estimation 
of the extent to which the obtained 
percentage of consistency between 
paired responses exceeds the percent- 
age which might be expected based 

on chance, chance probability being 
calculated from the marginals of the 
contingency table. Assume, for 
example, that in the sample 90% actu- 
ally watch TV and 10% do not, and. 
that 90 and 10 are the marginal per- 
centages both times we ask the ques- 
tion. Now assume that we randomly 


` pair first responses and second. 


responses rather than pairing two 
answers given by the same respondent. 
As discussed in Chapter 3, we can 
calculate the probability of agree- 
ment between pairs of response by 
chance from the marginals, multi- 


“plying .9 x .9 and .1 x .1 and adding 


the two to yield a probability of .82. 
This means that 82% of the pairs of 


_ response would agree simply by chance. 


When response distributions are lop- 
sided like this (e.g., when there 

is not approximately a 50-50 split 
between "yes" and "no" responses at 
each asking), the chances of agree- 


„ment become relatively high as illus- 
trated in this example. 


Thus the 
obtained percentage of agreement, 
even if it is relatively high, may 
be no higher than what we would 
expect if we simply paired by random 
assignment one subject's response 
with another subject's response 
instead of pairing two responses 
given by the same subject. (For 
those whose memories of statistics 
are jogged by the image of red and 
green balls in a jar, the probability 
of drawing two red'balls bedomes 
progressively higher as the’ number 
of red balls as compared tó green 
balls in the jar increases.) In 
interpreting kappa statistics then, 
we must bear in mind this question: 


liy 


To what extent are people responding 
so consistently to the question both 
times that their consistency is 
greater than we would expect by 
chance? As can be noted in the 
table, several questions which have 
very high consistency figures do not 
have large or statistically signif- 
icant kappas, while several ques~ 
tions with much more modest consSis- 
ency figures do have significant 
kappas. When a question yields an 
extremely high proportion of "yes" 
responses on both askings, it becomes 
very difficult to obtain a large 
kappa simply þecause the odds of 
agreement by chance alone are also 
high. - É 


We are not willing to say that kappas 
should be used as the ultimate cri- 
terion of reliability. We consider 
it impressive to obtain 90% or 
greater consistency figures even if 
those figures are not a significant 
departure from chance expectations. 
Such high figures indicate that the 
picture of the group is reliable and 
that individuals within the group 
are saying essentially the same 
things on two occasions a week apart. 
On the other hand, we can be even 
more satisfied when the consistency 
figure and the kappa associated with 
it are high, as in the last question 
` in the table about cleaning the floor. 
‘Similarly, we can be particularly 
suspicious of the utility of a ques~ 
tion if the consistency figure is low 
and the kappa is low as well, as is 
the case for the notably ineffective 
question about whether residents are 
allowed to hit people, In between,’ 
we have softer ground to stand on 
and must take into account all the 
information obtained, the percentage 
saying “yes” on each occasion, the ` 
percentage of responses that are 
consistent, and the extent to which 
this percentage deviates from chance 
expectation. i 


‘Overall, we would conclude that .yes- 
no questions appear to yield fairly 


reliable data. However, we would 
reemphasize the warning about acquies- 
cence, because saying "yes" on two ° 
occasions does not necessarily mean 
that "yes" is the valid response. 


Six multiple choice questions 
repeated across the forms also posed 
interpretive problems. Three of 
these questions which asked subjects 
to identify the extent of their activ- 
ity as “a lot," "sometimes," "not 
much," or "never" (or in one case 

"a lot," "some," "not many, " and 
"none" in reference to number of 
friends) yielded 71.9% consistency 
on the average. This figure is rela- 
tively high considering the fact that 
the probability of matching responses 
by chance is lower for a four-choice 
question than for a two-choice ques- 
tion. Indeed the kappas for these 
items ranged from .51 to .64 and 
were all significant at the .01 level. 
However, three cther questions, 
asking respondents how they felt 
about aspects of institutional life, 
yielded extremely low consistency 
(46.2 on the average), even though 
the kappas for two of the questions 
(.27 and .20) were statistically 
significant. It is impossible to 
determine whether this was the case 
because of the subjective and mood- 
dependent nature of the questions or 
because a greater. proportion of the 
sample was able to respond to these 
pictorial questions than to the -ver- 


' bal multiple choice questions. In 


any case, this format cannot be con- 
sidered successful. ! 


The remaining eight questions 
repeated on both forms were, iopen- 
ended. Four were factual in nature, 
calling for: name, spelling f name, 
birthday, and characterization of the 
state school. _ Roughly two-thirds of 
those asked these questions were 
responsive to the first d second 
administrations of each; of these 
63.3% were consistently able to give 
their first and last names correctly; 
42.9% were consistently able to spell 


their names; 63.6% consistently iden- 
tified their birthdays; and 61.9% 
consistently described the place in 
which they lived. This finding is 
rather discouraging, not only because 
many persons could not answer these 
basic questions, but also because 
those who did often gave incorrect 
information both times. On the other 
hand, the eriteria for correctness 
were rather strict. ` 


The remaining four open-ended ques- 
tions were each analyzed by response 
categories, which were based on 
respons given by both/ residents and 
attendantg. In these cases, both the 
total percéntage corisistent (i.e., 
either mentioning an item in a cate- 
gory on both askings or failing to 
mention it both times)and the per- 
centage mentioning something both 
times were examined. There were 
marked differences between the two | 
measures. On a question asking for 
an enumeration of. activities with 
friends, for example, the average 
total consistency figure across all 
eight categories was 70.7%, but the 
average percentage of respondents 
consistently mentioning something 
both times was only 18.1%. Virtually 
the same thing occurred for the other 
questions asked. For example, asked 
about what they were being taught in 
school, 69.6% of those who could 
answer both times referred to academ- 
ies:ofsome kind, but for each other: 
category of response, high total con- 
sistency was achieved by failing to 
refer toa category both times. 
There seems to be no satisfactory 
way of analyzing such questions, for 
consistency depends on both the 
verbosity of the respondents and the 
number of categories developed by 
the coders. Perhaps the major con- 
clusion to draw is that’ open-ended 
questions calling for enumerations 
yield very little information, par- 
tially because many mentally retarded 
persons cannot respond appropriately’ 
to them in the first place, and par- 


tially because when they do respond, 
they say very little. Due to the 
predominance of "no mentions," only 
9 of 35 kappas were significant. 
However, since the total consistency 
figures for all 35 response cate- 
gories of four questions averaged 
83.9%, one can at least conclude 
that it was relatively rare that a 
respondent mentioned something one 
time but failed to mention it \the 
other time. \ 


Judging from this sample, reliability 
of response from one week to the next 
can generally be considered margin- 
ally adequate, except for the subjec- 
tive yes~no questions mentioned pre- 
viously about rules and decision 
making and the picture-choice ques- 
tions about the quality of institu- 
tiorial life. Reliability did vary 

as a function of both question format 


“and question content, but for the 


most part answers to questions 
repeated a week apart were largely 
similar. 


Reliability Over Time— 
Institutionalized Adults 

In the adult institution sample, 
48 questions were repeated on both 
forms and could be examined for 
reliability. The questions were 
analyzed in three separate groupings 
according to question format: yes~no, 
either-or, and open-ended. 


The 29 yes-no questions were divided 
into those which elicited factual 
information and those which required 
a subjective response from the inter- 
viewee, with the expectation that 
reliability might be higher for 
factual questions. Analysis of re- 
sponse consistency across both sur- 
vey forms revealed that the average 
consistency for yes-no questions was 
81.2%. Contrary to expectation, 
there was ‘almost no difference in 
consistency between the factual and 
the subjective questions (80.6% and 
81.7%, respectively). Response con- 
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sistency across this set of questions 
ranged from 97.3% to 60.5%. The 
highest consistency figures were 
found for questions like "Are you. 
usually happy?" Do you have a tamily?" 


and "Do you know how to cook?" 


while the lowest consistency figure 
(60.5%) was obtained on the question 


"Are you usually sad?" another question 


for which "yes" can be considered a 
socially undesirable response. 


There was a definite tendency for 
the majority of interviewees to con- 
sistently give "yes" responses ‘to 
many questions (21 of 29). It ap- 
peared that the reliability figures 
for this sample, as was the case for 
institutionalized children, were 


inflated by a tendency to acquiesce 
on yes-nd questions. Reliability 
was highest for those items for 


which "yes" was a socially desirable 
choice. However, even if such a 
response set were not a significant 
factor in the high reliability fig- 
ures obtained, it cannot be assumed 
that consistent responses .to these 
yes-no questions necessarily repre- 


. sent accurate or valid information. 


Thus the picture of this sample 
based on their responses to these 
questions appears to be primarily 
positive, with the majority of the 
interviewees seeing themselves as 
usually happy or possessing a number 
of baSic adaptive skills such as : 
reading, writing, keeping house, 
counting money, and cooking. This 
picture may have been influenced by 
a tendency to acquiesce or to give 
socially desirable responses as al- 
ready discussed, and indeed we will 
demonstrate later that attendants 
are not so ready to agree that 
clients have basic adaptive skills. 
However, the responses to questions 
such as, "Are you usually sad?" and 
"Do you have any problems?" seem to 
suggest that many clients can say 
"no" to deny having difficulties. 


there were four verbal either-or 
questions repeated on tne two alter- 
nate surveys consisting of two pairs 
of questions with the order of op- 
tions varied: "Are you usually happy 
or sad?" vs. "Are you usually sad or 
happy?" and "Are you usually by = 
yourself or with other people?" vs. 
"Are you usually with other people 

or by yourself?" Average consist- 
ency for these four questions was 
again relatively high (82.2%) and was 
comparable to the average consistency 
obtained for yes-no questions. A 
substantial majority of respondents 
consistently indicated that they were 
usually happy im response to both 
happiness questions. However, there 
was a tendency for more interviewees 
to consistently answer "sad" when this 
alternaitve was given second (12.0%), 
‘than when it was the first option (2.9% 
consistently reporting "sad"). 


The relationship between the two 


- questions of the second either-or 


pair was more complicated, although 
a similar pattern emerged. The 
majority _ of respondents consistently 
replied to both questions that they 
were usually with other people as 
opposed to being by themselves. As. 
with the previous question pair, the 
percentage who consistently chose 
the alternative “with other people" 
was slightly higher when this choice 


.was presented as the-second option 


in the question than when it was 
first (50.0% vs. 46.7%). In 
addition, the percentage who consist- 
ently ‘chose the "by yourself" alter- 
native doubled when this option was 
presented last (33.3% vs. 16.7%). 

For this question, "Are you usually 


‘ with other people or by yourself?", 


the effect of consistently choosing 
the last alternative was to increase 
the overall percent of respondents 
who answered the question consist- 
ently both times to 80.0% for this 
question as compared to 66. 7% for 

the alternative question. The obser- 
vations made from these two either- 
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or question pairs seem to indicate 
that a small portion of interviewees 
are probably influenced more by the 
order of the either-or alternatives 
than by the actual content of the 
response alternatives. j 


The last 15 questions on both surveys 
were open-ended. The first seven 
open-ended questions ‘asked the resi- 
dents to give factual \information 
concerning their names), birthdates, 
and current addresses. Responses to 
these questions were coded either 
correct or incorrect based on records 
kept on the subjects. ' On the average, 
51.9% of the residents who. responded 
to be juestions consistently gave 
correct answers, Residents were most 
likely to give correct information 
about their dates of birth: 77/4% 
corréctly identified the month they 
were born in, 72.2% consistently gave 
the correct date, and 58.8% were able 
to give the right year both times. 

In comparison, only one-third (33.3%) 


„Of the residents could correctly 
‘give, their full present address con- 


sistently, while slightly more than 
half (56.7%) were able to correctly | 
give their first and last names at 
both interviews (only first or last 
names were scored as’ partially } 
correct). Only 25.0% of the resi- 
dents Gould consisténtly spell their 
names (orally) when asked, making 
this the most difficult question of 
the seven.’ A greater number of sub- 
jects (40.0%) were able to cor- 
rectly write their names both times. 
Five of the 15 were coded with pre- 
sent response categories, and the 
average consistency for these ques- 
tions was 83.3%. This figure would 
have been higher except that response 
consistency for one question, "What 


. kind of place is this?", was excep- 


tionally low, with 55.0% giving con- 
sistent descriptions and only 30% 
consistently identifying the place 
where they lived as a state school. 
A clear majority of the respondents 


consistently indicated that someone 


' category. 


other than themselves made ‘the rules 
-where they lived (92.0%) and decid~ 
ed what chores they would do (75.0%). 
In contrast, the respondents were 
almost equally divided on the ques- 
tion "who decides how you spend 

your money?", with 50.0% consistently 
answering "someone else" and 43.8% 
consistently indicating that they 
made the decision themselves. Total 
reliability figures for these three 
questions were uniformly high:° 96.0%, 
83.3%, and 93.8%, respectively. 

These findings indicate that most 
subjects who can answer giVe reliable 
answers to open-ended questions con- 
cerning who makes decisions about 
their everyday activities. In\addi- 
tion, 90.8% of the subjects consist- 
ently answered the question, "Ho 
many people sleep. in your bedroom? 


The remaining three open-ended ques- 
tions assigned each resident's an- 


-swer to one of several response cate- 


fi 


gories made up on the basis of the 
responses given by both residents 
and attendants. ‘The average consist- 
ency for this set of 26 open-ended 
response ‘categories was 87.4%, the 
highest average consistency, for any | 
of the question groups analyzed- ` 
However, the consistency percentages" 
for the response category questions 
were inflated by the large number of 
categories whose high consistency 
was achieved by clients' repeatedly 
failing to mention anything in the 
For example, on. one ques- 
tion which asked the resident to 
name the person who helps him/her 

the most, there was only one category 


' (cottage staff) for which the most 


-common esponse combination was men- 
tioning!’ that category both ‘times. 
For the other eight response cate- 
gories for this questic the most 
common response combinat.on was not 
mentioning the category either time. 
Therefore, while the. average total 
consistency figure across all nine 
response categories was high (81.5%), 
the average percentage of: respondents 


H 


6.8 


consistently mentioning something 
both times was very low. 


This same pattern was found for the 
remaining two response category ques- 
tions. When residents were asked 
who they usually talked to about 
their problems, the most consistently 
mentioned response category was "cot~ 
tage staff" (36.4%). Four out of the 
remaining six categories were men- 
tioned by only a few residents, 
making the average percent of respond- 
ents mentioning a category both times 
again quite low (9.1%). On the final 
question of this group, "If you had 
one wish, what would you wish for?" 
no response category was mentioned 
very frequently on both occasions. 
The most frequently and consistently 
mentioned wishes were to be with 
their families (27.3%) and to have 
material items (13.6%). Consistency 
in responding to this question was 
achieved. primarily by failing to men- 
tion a response category both times. 
Thus open-ended questions, which 
could not be. answered by many subjects 
in the first place, were reliably © 
answered, but rarely generated con- 
sistently mentioned information. 


Overall, for this sample of insti- 
tutionalized adults, response con- 
sistency for questions repeated at 
a one-week interval averaged over 
80%, providing interviewers with 
\answers that were reasonably reli- 
‘able. There was little variation in 
EN consistency associated with 
the three different question formats, 
However, there was some indication 
that high consistency on yes-no 
questions may have been gained in 
pakt through acquiescence, that some 
subjects prefer the second alterna- 
tive on either-or questions, and 

that consistency on open-ended ques- 
tiong was gained primarily by fail- 
ing mention something both times 
rather than mentioning something 
twice. 


Individual Correlates of Response 
Reliability 

As was done in analyzing respon- 
siveness as an individual character- 
istic, an analysis was conducted to 
determine what accounts for individ- 
ual differences in reliability of 
response from one week to the next. 
In the children's institution sample, 
a reliability score was formed based 
on the number of. instances of con- 
sistent responses to 17 repeated yes- 
no items divided by the number of 
item pairs answered appropriately. 
This index was calculated only if 
subjects had been able to answer. 
half or more of the 17 repeated item 
pairs. A comparable index was formed 
for institutionalized adults based 
on 29 yes-no items, Then, in each © 
sample, this reliability score, which 
could theoretically range from 0% to 
100%, was correlated with IQ, age, 
sex, responsiveness to the entire 
Form A interview schedule, and an 
acquiescence index, on which subjects 
were given a score of l as opposed 
to 0 if they said "yes" to both "Are 
you usually happy?" and "Are you usu- 
ally sad?" We expected that sub- 


‘jects with higher IQs and higher 


responsiveness scores indicative of 


verbal skill would be more likely 


-than lower IQ ‘and less responsive 


subjects to give consistent responses 
on two occasions. At the same time, 
we predicted that subjects who tended 
to acquiesce might be likely, on yes- 
no items at least, to achieve high 
reliability scores simply by saying 
"yes" a great deal. No specific - 
predictions were formulated about 

the relationship between age and sex 
and reliability of response. 


Reliability scores in’ both samples 
were normally distributed. In the 
children's sample, these scores 
ranged from 59% to 100% (mean = 85.8%, 
SD = 11.1%) for the 44 of 52 subjects 
for whom they were calculated. Simi- 
larly, in the adult sample, 40 of 58 
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subjects had enough data to calculate 
scores, and scores ranged from 58% to 
97% (mean = 76.1%, SD = 10.2%). Inter- 
estingly, then, no ore was inconsist- 
ent on more than half! of the, items 

for wūich first and second adminis- 
tration answers were javailable. 





Table 6.2 presents the simzle corre- 
lations between indiyiaual character- 
istics and the reliability of 
response score. These correlations 
are very difficult fo interpret, 
particularly since jthe findings in 
the two samples do/not correspond 
well, In the children's institution 
sample, contrary to expectation, 
lower IQ subjects) were actually more 
likely to. give réliable responses 
than higher IQ subjects. Other cor- 
relations were small and inconsistent 
in direction, but there was at least 
a tendency for, more acquiescent sub- 
jects to display more reliability of 
response. In’ the adult institution 
sample, by contra..t, there were small 
positive associations between both 

IQ and responsiveness and reliability, 
the relationship between responsive- 


ness and reliability being statis- 
tically significant. In this sample, 
there was essentially no relation- 
ship between acquiescence and relia- 
bility. About the only consistency 
across samples was a very slight 
tendency for ‘females to provide a 
greater proportion of reliable 
responses than males did. 


These Simple correlations do not `` 
take into account the fact that some 
of the individual characteristics 
involved in the analysis are inter- 
related. As noted in Chapter 5, IQ 
and responsiveness were positively 
associated in both samples. More- 
over, there is a consistent tendency 
for higher IQ persons to be less 
likely to acquiesce as indicated by 
responding "yes" to both happiness 
items. The correlation between IQ 
and acquiescence was .40.in the 
children's sample, and .31 in the 
adult sample. Therefore, partial 
correlations, which allow for exam- 
ining a relationship. between two i 
variables.with the effect of a third 


controlled, were conducted to attempt 


a o a 


Table 6.2: 


Child Institution 
Reliability 





Individual Correlates of Reliability of Response 


Adult. Institution 
Reliability 


*Sigr‘ficant at the .05 level : . 
**Significant at the .01 level . o 
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to unravel the independent impacts 
‘of IQ, acquiescence, and responsive- 
ness on the reliability measure. 
However, these analyses did not clar- 
ify the findings. They left the 
pattern of correlations in Table 6.2 
essentially unchanged. 


Thus, we must conclude that the 
extent to which mentally retarded 
persons give consistent answers on 
two occasions is essentially unpre- 
dictable based on the information 

we had available. It is still pos- 
sible that opposing forces were at 
work; that, on the one hand, higher 
ability subjects were likely to be 
consistent in their responses but 
that, on the other hand, acquiescent 
_ tendencies inflated consistency 
scores for lower ability subjects. 
Unfortunately there were not enough 
items in formats other than yes-no 
to allow us to see if reliability of 
response where acquiescence does not 
operate is more predictable. 


Summary and Conclusions 

‘ This chapter has been concerned 
with one aspect of the issue of 
whether answers given by mentally 
retarded persons can be considered 
useful information. In both insti- 
tution samples, repetitions of ques- 
tions a week ‘apart provided evidence 
of response reliability. In both 
samples, it was found that answers 
given on two occasions were generally 
consistent over 80% of the time. 
Figures for yes~no, either-or, and 
open-ended questions with response 
categories were generaily very simi- 
lar and high. The lowest consisten- 
cy fiqures were obtained for the mul- 
tiple choice questions used in the 
children's institution sample, es- 
pecially those questions about sat- 
isfaction which asked respondents to 
choose among happy and sad faces. 
At the same time, as kappa statistics 
associated with these items suggested, 
it is more difficult to obtain con- 


sistency by chance on four-option 
questions. Thus, although the con- 
sistency figurés for these items were 
relatively low, it appeared that re- 
spondents were not just answering in 
a random way. 


While we might be more gratified by 
consistency closer to 100%, we must 
conclude that what mentally retarded 
persons say one week is likely to be 
quite similar to what they say the 
next week. Most of our questions 

were factual in nature so such con- 
sistency should be expected. In- 
terestingly, however, an analysis in 
the adult institution sample of fac~ 
tual versus subjective yes-no ques- 
tions did not reveal lower reliabil- 
ity of responses to subjective ques- 
tions, despite the fact that such re- 
sponses might be predicted to be more 


' mood-dependent and changeable’ from ` 


week to week. 


As we have noted throughout this chap- 
ter, however, reliability of response 
figures was very difficult to. 
interpret. In response to yes~no 
questions, an unthinking endorsement 
of items would be scored as consist- 
ency of .responses in-exactly the 

same way as a thoughtful provision 

of "yes" responses on both occasions. 
Given our findings of high rates of 


acquiescence to yes-no questions in 


retarded samples, as well as the 
positive association between the 
percentage of the sample endorsing 
an item and. the consistency figure | 
for that item in the present analy-/ 
sis, we must conclude that relia- / 
bility figures for yes-no questions 
are inflated. At the’ very least, / 
we must be cautious not to construe 
them as evidence that answers given 
a week apart express a consistent 
body of "information" about the /lives 
of the respondents. We can conclude 
that the picture of the group, and 

of the individual within it, bdb- 
tained from yes-no questions does 
not change much from week to week, 

as long as we recognize that these 
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"pictures" may not be faithful repre- 
sentations of reality. 


Much the same can be said of relia- 
bility figures for open-ended ques- 
tions. We suspect that answers to 
them underrepresent the extent to 
which mentally retarded people are, 
for example, involved in various 
activities, in the same way that 
yes-no questions probably overrepre~ 
sent extents of involvement. While 
we did not have a completely satis- 
factory way of analyzing consistency 
for these questions, it was apparent 
‘that few responses were given to 
such questions and consistent men- 
tion of items was a rare phenomenon 
simply because mention of items was 
a rare phenomenon. Unfortunately, 
the reliability of .such items was 
heavily influenced by the Number of 
categories that the coders chose to 
construct. The fact that reliability 
for categories with frequent mentions 
tended to be lower than reliability 
for categories with virtually no 
mentions is not a healthy sign. 


As for items with structured choices, . 


they were associated with relatively 
low reliability figures. While 
either-or items fared well despite 
the fact that they were subjective in 
nature, verbal multiple choice items 
calling for an indication of extent 
of activity were only marginally 
reliable. Pictorial multiple choice 
questions about satisfaction with 
aspects of institutional life were 


not responded to ir -he sam way on 
two occasions, even though obtained 
consistency figures exceeded chance 
probabilities. These figures, too, 
are difficult to interpret. Subjec- 
tive feelings about the institution 
might well change from week to week 
as a function of mood and intervening 
experience, 
given on both occasions. 


All in all, then, our analysis of 
reliability is encouraging in the 
sense that answers can generally be 
counted on to be the same from week 
to week, but discouraging in the 
sense that we are not sure how to 
interpret even high reliability 
figures. Moreover, we were unsuc- 
cessful in identifying which persons 
are most likely to provide reliable 
data. This uncertainty. will, however, 
be reduced as we turn to issues of 
response validity. In the next chap- 
ter, we will take up the question of 
whether clients’ responses agree with 
responses to the same questions pro- 
vided by attendants and parents. In 
this analysis, we will be able to 
look more directly for evidence of 


‘systematic biases in the responses 


given by mentally retarded persons. 
Such biases as acquiescence, selec~ 
tion of the.-last option on either-or 
or multiple choice questions, and 
failure to mention things mentioned 
by significant others, to the extent 
that they occur,.may cast the reli- 
ability figures presented here in a 
different light. 
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TO WHAT EXTENT ARE ANSWERS VALID? 


The two previous chapters have 
dealt with responsiveness of men- 
tally retarded persons to interview 
questions and with the reliability 
of their responses. While it is 
clear that these factors are im- 
portant, it is equally clear that 
even reliable responses to questions 
are useless if they are not valid. 
That is, it is pointless to elicit 
responses which do not accurately 


reflect the attitudes and behaviors 


of the mentally retarded persons 
interviewed. We have devoted con- 
siderable attention to evaluating 
the validity of responses by our 
mentally retarded respondents. . 


validity has been to compare the 


Our main- strategy for assessing J: 


sponses of adults who should be 
knowledgeable about the mentally 


rd 


‘retarded interviewees. Thus, when 


we asked a child "Do you go to. 


church?" we asked the child's/parent 


or cottage attendant if the child 
goes to church and noted the per- 


“centage of cases in which the two 


answers agreed. Clearly, there are 
instances in which the parent or 
attendant will not be in a good 
position to provide information re- 
garding the mentally retarded respond- 
ent. Especially when the attitudes 
of retarded persons are at issue, 
attendants' or parents' opinions may 
not be accurate or even meaningful - 
Then, too, we must recognize that 
even on points of fact parents, and 
perhaps especially attendants, may 
simply not be knowledgeable, and 
their answers, like those of re- 
tarded persons, may be biased. None- 


/ theless, we assume that the degree 


of. correspondence of the responses 
of children and their significant 
_others. is a useful (although admit- 
tedly imperfect) indicator of re- 
sponse validity. 


A second strategy for evaluation of 
response validity has been to ask 
questions to which the correct answer 
is known. This technique reduces 

the ambiguity inherent in the first 
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strategy. If, in response to such 
questions, mentally retarded persons 
are able to provide accurate infor- 
mation, we have sound evidence of | 
their ability to give accurate re- 
sponses to at least some interview 
questions. 


Our use of these techniques in the 
course of this project was extensive. 
Therefore, a discussion in detail 

of all validity data would be im- 
possible. Fortunately, the analyses 
of agreement in the different sam- 
ples yielded strikingly consistent 
results. Thus, we will discuss in 
detail the comparisons of children's 
samples, using that data to illus- 
trate the major conclusions. Sub- 
sequently, agreement data from the 
additional samples will be discussed, 
citing only those data which either 
contradict the conclusions derived 
from the community children's sample 
or which add novel insights. Then, 
the validity data derived from the 
use of questions to which answers 
“are known will be discussed. 

Finally, we will look for differences 
among mentally retarded persons in 
their tendencies to give valid re- 
sponses as indicated by agreement 
with significant others. 


Community Children 


OVERALL AGREEMENT : 

Sixty-one questions were askel 
of both community children and théir 
parents, and the degreé of corre- 
spondence between their responses) 
was evaluated as a check of response 
validity. Average agreement for 
this set of questions was 64.5%; 
that is, Parents and their retarded 


children were in agreement about two- 


thirds of the time. Overall agree- 
ment and a breakdown of agreement 
according to question type are de- 
picted in Table 7.1. Clearly 
agreement did vary as a function of 
- question type, but as the discrepant 
figures for two different types of 


_ by yourself?"). 


multiple choice questions suggest, 
it is necessary to look more closely 
at both question format and question 
content in order to understand what 


makes agreement likely to be high or 


low. 


YES-NO QUESTIONS 
Yes-No Questions With Words Only 


Average Agreement 
Questions posed of non- 


institutionalized children and 

their parents included 22 simple 
verbal questions, most about client 
activities, requiring "yes" or "no" 
responses. The responses of clients 
and their parents were compared; data 
for these 22 comparisons are summa- 
rized in a mean contingency table 
(Table 7.2). 


Children and parents tended to agree 
in their responses to yes~no ques- 
tions; on the average, 65.7% of 
comparisons yielded child-parent 
agreement. As can be seen, agr2e- 
ment tended to resuit'primarily 
because both clients and parents 
responded "yes." Because clients 
favored "yes" slightly more than did 
their parents, the majority of. dis- 
agreements occurred when the client 
said "yes," and the parents said 
"no," suggesting that acquiescence 
by retarded persons may threaten the 
validity of their answers. 


Differences in Agreement Due to, 
Question Topic 

The use of averaged data can ob- 
scure important factors underlying _ 
responding. In fact, agreement 
varied greatly from question to 
question, ranging from 92.8% ("Do 


_you watch TV?") to 40.8% ("When 


your're not at school,are you usually 
In pursuit of ex- 
planations for this variation, we 
examined in some detail the differ- 
ences in agreement rates within 

this group of questions. First, 

it would appear that relatively 
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\ | Table 7.1: Agreement by Question Type 2 
Question type i # of Questions p Agreement : 
Yes-no (words only) 22 , 65.7 
Yes-no (with lead-in & pictures) 10 
Either-or (Bib jective: words only) 7 
Either-or\ (subjective; with.pictures) 4 








Mustiple choice (discrete) 
Multiple choice (quantitative) A 
Open-ended (factual) O 4 
* Open-ended (coded responses) _ 3 
Total 61 64.5 


o n__n 


*Calculated by averaging the average agreement scores across response categories 
for the three questions 


simple and concrete questions yield go to church?" was. 78.2%, as opposed 
higher agreement rates than do more to 56.4% for "Most of the time, 
complex and abstract questions. is it up to you to decide what time 


For example, agreement for "Do you you go to bed?" 





Table 7.2: Mean Contingency Table for 22 Verbal Yes~No Questions 


i 





Agreement = 65,7% 
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Five questions which asked about 
the client's participation in various 
activities had an average agreement 
rate of 82.3%, in contrast to 66.5% 
‘for a group of séven more complex 
and abstract questions déaling with 
training in specific skills ("Is 
anybody at school teaching you about 
money now?" iS an example of the 
latter group of questions). 


Second, it appears that some topics 
include built-in response biases 
which influence agreement. For 
example, three questions which in- 
quired about client's possession of 
particular skills ("Do you know how 
to read books?") resulted in agree- 
ment lower than that for any other 
topic area--54.3%,. It seems un- 
likely that this is attributable to ~ 
the difficulty of these questions; © 
rather, a specific bias appears to 

be in operation here. That is, 
clients frequently claim skills- 

which their parents deny they possess. 
On the average, 81.1% of clients 

and only 52.7% of parents answered 
"ves" to the skills questions, so 
that fully 37.0% of response com- 
parisons were disagreements in which 
clients claimed skills and were, 
contradicted by their parents. 


3 


Is this a matter of client's overes- 
timating their skills or parents 
underestimating the skill levels of 
their mentally retarded children? 
Although we cannot be certain, the 
former possibility seems more likely 
in.this case. In responding to 
skills questions, clients probably 

_ become concerned with favorable 
self-presentation so that a desire 
to appear compecent encourages | 

an ate®irmative answer. Yes" answers 
probably also reflect a more general 
tendency ‘~v ord acquiescence, but 
concern wi.* “ €-presentation is 
probably whav nts for the par- 
ticularly low agreement on these 

| skills questions. 
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Differences in:agreement as a func- 


tion of parents' responses 
Another pattern appeared in 


variations of agreement rate within 
“the 22 yes-no questions. 


Specifi- 
cally, agreement was highest for 
those questions to which the parents 
are most inclined to respond "yes." 


: This is to be anticipated in light 


of an observation made throughout 

the course of this project that men- 
tally retarded respondents tend to 
acquiesce, or to respond "yes" re- 
gardless of question content. To the 
extent that acquiescence on the part 


of the clients occurs in response 


to yes-no ‘questions, higher agreement 
should be observed on those questions 
for which the response of significant 
others is "yes." Here agreement , 
would not indicate response validity 
as much as a happy coincidence of 
response bias and reality. 


To test that possibility, this group 
of questions was divided into the’ 11 
questions which parents tended most 
often to answer "yes" (average of . 
88.9% "yes" responses by parents) ‘and 
the 11 questions to which parents 
were least inclined to respond "yes" 
(average 44.6%). Comparison of the 
agreement” rates for these two groups 


of questions is depicted in Table 
7.3. As can be seen, average 
agreement for "high yes" questions 
was 75.7%, as opposed, to 55.6% for. 
the "low yes" questions. Looked at 
another way, the percentage of 
parents responding "yes" to these 

22 questions correlated, .80 with 
agreement figures for the questions. 
‘This comparison suggests that clients 
tend to indiscriminately respond 
"yes" to yes-no questions, and that 
client-parent agreement is to some 
extent artificially controlled by 
the proportion of "yes" responses 
given by parents. Thus overall 
agreement rate provides an inflated 
estimate of validity when the ques- 
tions involved are predominantly E 
answered "yes" by significant others. 
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Table 7.3: Child-Parent Agreement as a Function of Parent's Tendency 
to Say "Yes" 








11 Questions, Low Proportion of Parent "Yes" Responses 


Client Significant Other 





No 11.6% 34.2% 
Yes 33.0% 65.7% 
° |. 55.3% 44.6% 
a Agreement = 55.6% 


Client ` Significant Other 





Agreement = 75.7% 








A set of questions to which 50% of mentally retarded individuals tend 
parents' responses were "yes" would to elicit less valid responses than 
provide a better test of validity of do easier questions (to the extent 
yes-no ‘questions. that comparison with their parents' 


responses constitutes an adequate 
measure of validity). Second, spe- 
cific topic areas (for example, ques- 
tions about skills and competencies) 
may involve biases which tend to 


Conclusions 

In short, these data suggest 
three conclusions. First, cogni- 
tively demanding questions posed of 
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systematically alter responses in 


Such a way as to seriously reduce 


validity. Third, average agreement 
rate is an overestimate of validity 
for this group of questions, for 
which the parents' response was "yes" 
66.8% of the time. Where parents 
respond "yes" with frequency, acqui- 
escence on the part of retarded re- 


spondents inflates agreement figures 


but not necessarily validity. It is 
likely, in light of this observation, 
that the actual validity of these 
yes-no questions is low enough that 
their usefulness as a means of 
gathering information from mentally 
retarded persons is questionable. 


Yes-No Questions 

with Lead-in and Pictures 
Interviews with community chil- 

dren included 10 yes-no questions 


.which were preceded by a verbal 


lead-in and which used pictures to 
clarify questions. These 10 ques- 
tions concerned chores, and the ver- 
bal lead-in read: “Here are some 
pictures of different kinds of chores 
some people do. I want you to tell 
me if you do any of these chores at 





‘because both client and parent re- 


Sa 
~~. 


home." The questions were simple 
yes-no questions such as "Do you set 
the table?" and "Do you make beds?" 
each question being accompanied by 

a picture of youngsters engaging in 
the chore involved in the questions. 


Average agreement 


Data for the 10 questions were 
averaged and are depicted in a mean 
contingency table (Table 7.4). 
Average agreement for these questions 
was 71.5%. Nearly two-thirds of the 
response comparisons were agreements 
in which both child and parent re- 
sponded’ "yes." Inconsistencies were 
divided between cases in which the, 


‘client responded "yes" and the parent 


"no," and the converse; thus, there 
was not strong evidence of acquies-. 
cence for the set as a whole.. In 
short, agreement for this class of 
questions was relatively high, some- 
what higher than for the verbal yes- 
no questions, and tended to result 


sponded "yes" to most questions. 


Table 7.4: Mean Contingency Table for 10 Yes-No 
Questions with Lead-in and Pictures 








Responses of Clients 















Responses of Significant Others 


Agreement = 71.5% 
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pifferences in agreement 


There was relatively little 
variability in agreement among these 
questions. For 9 of the 10, the 
range of agreement was only from 
65.5% to 85.5%. It may be argued 
that the uniformly high agreement for 
these questions results from the fact 
that the "valid" answer to most of 
them (as reflected by the parents' 

_ response) was "yes." However, agree- 
ment for "Do you cook on the stove?" 
was only 47.2%, by far the lowest 
agreement for the set. It is not 
coincidental that this question also 
had the lowest percentage of "yes" 
responses from parents (52.7%). 
Clients tended to respond "yes" to 
this question (65.4%), with the 
result ‘that 32.7% of the comparisons 
were inconsistencies in waich clients 
responded "yes" while their parents 
were responding "no. This inter- 
pretation is further supported by a 
comparison with the question from 
this group which had the highest pro- 
portion of "yes" responses from 
“parents: "Do you pick up stuff 
around the house?" Ninety-six per- 
cent of parents answered "yes," and 


_ the agreement, rate was 85.5%, highest 


for this set of 10 questions. All 
agreements in this case resulted be- 
cause both client and parent respond- 
ed "yes." For this group ot ques- 
tions, the correlation between pro- 
portion of parents responding "yes" 
to a question and agreement for that 
eee was .97. ig 


Conclusions 

The high average agreement for 
these yes-no questions appears to 
overestimate response validity. 
Average agreement is high because 
the apparently appropriate answer to 
most of these "chores" questions is 
"ves." When parents give a high 
proportion of "no" responses, client- 
parent agreement drops accordingly, 
as it did on verbal yes~no questions. 


EITHER-OR quesrroN 


Either-or Gusaetone with Words Only 


Seven questions asked of commu- 
nity children and their parents were 
simple verbal questions posing two 
response alternatives. "Are you 
usually happy?" for instance, is the 


complement of the above example. 


Average Agreement j 
Agreement for this group of ques- 
tions ranged from 50.0% to 86.0%; 
average agreement was 65.6%. Average 
agreement was thus comparable to that 
for yes-no questions using words only 
(65.7%), but- the range for those yes- 
no questiéns was much greater (40.8% 
to 98.2%). To what may this decreased 
variability for the either-or ques- 
tions be attributed? As has been 
pointed out, the upper end of the 
distribution of agreement rates for 
yes-no questions appears to be an 
“overstatement of the validity of 
those questions, due to acquiescence 
by mentally retarded respondents and 
the fact that the payent' s response 
to those: questions is typically "yes.' 
It seems likely that agreement rates 
for either-or questions are more uni- 
form because no such biases operate 
in these| questions to generate some 
spuriously high estimates of validity. 
Indeed, we’ have found no reason to 
suspect that agreement rates for 
either-or questions are anything 
other than an accurate indication of 
the validity of responses to these 
- questions, for disagreements were 
generally mixed in nature. Despite 
roughly equivalent agreement rates 
for the two types of questions, then, 
it seems likely that either-or ques- 
tions generate responses of greater 
validity. Agreement for yes-no 
questions is inflated simply because 
our survey unfortunately included a 
predominance of questions which were 
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commonly answered "yes" rather than poe 
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Differences in agreement 

No difference in agreement due to 
the reversal of order of the response 
alternatives was apparent. The rates 
of agreement for two of the three 
pairs did not differ (50.0% vs. 51.0% 
in one pair, and 72.5% vs. 73.2% in - 
the other). In the third pair, the 
"happy or sad" example stated above, 
agreement figures for the two forms. 


were 86.0% (with "happy" first) and 
72.0% (with "sad" first). No expla- 
nation for this difference is readily 
apparent. Where we have observed 
order effects on either-or questions 
in our interviews, they have typi- 
cally taken the form of endorsement 
of the last-mentioned alternative 

by mentally retarded respondents. 
That bias, however, does not account 
for the agreement difference observed 
inthis pair of questions. In fact, 


more clients said they were usually 
happy when "sad" was the last men- 


tioned alternative. It would appear 
that agreement for these either-or 
questions does not differ systemati- 
cally as a function of the order in 
which alternatives are presented. 


There do, however, appear to be dif- 
ferences in agreement due to question 


. topic, just as there were for yes-no 


questions. While it is difficult to 
predict which topics will yield 
agreement with significant others, 
variability in agreement as a func- - 
tion of question topic at least 
suggests the need for caution in - 
interpreting data: gathered in inter- 
views with mentally retarded persons, 
and also suggests that we be conserv- 
ative in generalizing demonstrations 
of validity across content areas. 


Either-or Questions With Pictures 
The group of questions asked of 
community children and their parents 


included two pairs of either-or ques- 
tions which used pictures as the 


response alternatives. The members 
of each pair involved the same con- 


ranged from 41.5% to 74.1%. 


. siveness. 


tent but reversed the order in which 
the response alternatives were pre- 
sented. For example: "This picture 
(point) shows a boy/girl who is 
happy, and this one (point) shows a 
boy/girl who is sad. Which picture 
shows how you usually feel? Point 
to the picture." 


Average agreement for these either- 
or questions with pictures was 58.1%. 
Agreement for the individual questions 
Again, 
it appears that agreement varies more 
or less unpredictably across topic 
areas. Agreement was much higher for 
the two happy-sad questions (74.1% 
and 66.7%) than for the other two 
qu.stions, which asked whether the 
client spends free time alone or with 
otners (41.5% and 50.0%). Disagree- 


ment on the latter pair of questions 


tended to result because clients 
chose the "alone" pictures, while 
parents responded "with others." The 
agreement rate for these four ques- 
tions did not vary systematically as 
a function of the order in which . 
response alternatives were presented. 
Moreover, the’ use of pictures as’ 
response alternatives did not improve 
agreement over that for verbal either- 
or questions on the same topics. If 
any difference can be said to exist, 
it is that either-or questions with 
words yield somewhat superior agree- 
ment (64.8% vs. 58.1%). This appar- 
ent decrease in validity for picture 
questions.may be due to the fact that 
the use of pictures increases respon- 
Many individuals who do 
not respond_verbally can.respond by 
pointing to pictures. The response 
patterns of these less verbal, lower 
IQ respondents may be largely random, 


_xreducing validity. We do in fact 


have some evidence that the responses 
of lower IQ respondents tend to‘be 
less valid (see discussion at end 

of’ chapter). 
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Conclusions 

Average agreement for either-or 
questions is similar to agreement for 
yes-no questions. Because agreement 
figures probably give an overestimate 
of the validity of yes-no: questions,. 
we can conclude that either-or ques- 
tions generally yield more valid 
responses. There is no evidence 
suggesting that agreement is not an 
accurate indicator of validity for 
either-or questions.” Agreement does 
not appear to be affected by revers- 
ing the order of presentation of re- 
sponse alternatives, although it 
does vary as a function of question 
topic. Finally, the use of pictures 
as response alternatives apparently 
decreases validity of responses to 
either-or questiongggomewhat, prob- 
ably because less capable. clients ` 
can then provide answers. 


MULTIPLE CHOICE QUESTIONS 
Eleven questions posed of non- ` 
institutionalized children and their 


‚parents were multiple choice ques- 


tions, involving a cnoice between 
either three or four response alter- 
natives. Average agreement was 57.0%. 


‘These questions differed in question i 


format: some were three-choice and 
some four-choice questions, some 

used words only and some words plus 
pictures in depicting response alter- 
natives, and some of the questions 
were follow-ups to yes~-no questions. 
However, there were no systematic 
differences in agreement due to any 
of these format considerations. 


Agreement for multiple choice ques- 
tions did, however, differ according 
to the type of response alternatives. 
Six multiple choice questions gave 
discrete alternatives: about matters 
of fact. ("Do you live in a house, an 
apartment building, a tráiler house, 
or a duplex?"), while five used 
quantitative dimensions as response 
alternatives ("How many friends do 
you have: a lot, some, not many, Or 


ik 


© correct answer. 


none?"), Average aqreement for the 
discrete questions was 77.9% (range 
70.5% to 85.5%); average agreement / 
for the quantitative multiple choice 
questions was only 24.0% (range 25.9% 
to 39.0%). As is readily apparent, . 
not only does average agreement for 
the two groups of questions differ, 
but their renges do not come close 
to overlapping. Clearly, agreement 
was superior for the questions 
offering discrete alternatives. ` Be- 
cause quantitative responses. require 
finer discriminations, this trend 
might well be present in normal pop- 
ulations as well. However, the fact 
remains that average agreement was 
very high for the discrete questions 
and unacceptably low for those 
offering quantitative alternatives. 
This suggests that discrete multiple 
choice questions may be useful in > 
interviewing mentally retarded per- 
isons, but it would appear pointless 


Ito ask mentally retarded respondents 
| to make quantitative judgments, 

i given the absence of evidence for 
| validity of their responses. 


Conclusions 

Multiple choice questions 
offering discrete response alterna- 
tives yield very high client-parent 
agreement, Agreement for questions 
using quantitative alternatives is 
so low as to make these questions 
useless in obtaining information 
from mentally retarded respondents. 
No conclusion can be offered re- 
garding format differences such as 
number Jf options and-method of 
presentation of response alternatives. 


` 


OPEN-ENDED QUESTIONS 


Gren sended: Factual Questions re 


Questions allowing comparisons 
of the responses of children and 
their parents included four open- 
ended questions having a single 
» On the average, 
children matched their parents 64.8% 


7.9 
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of the time. It is somewhat dis- 
couraging that validity was this low 
for.simple questions requesting fac- - 
tual information which should be 
readily available to mentally re- 
tarded respondents. Aareement was 
only 42.5% for the question "Counting 
you, how many people live in your 
house right now?" The other three 
questions asked the client's age, 

the starting time for school, and 
how the child gets to school; agree- 
ment figures for these quéstions 

were 75.0%, 56.3%, and 85.4%, re- 
spectively. In short, it appears 
that open-ended questions requesting . 
factual information yield responses 
of varying validity, depending on 

the questions. Apparently ques- 
tions involving number and time 
concepts were difficult. The dis- 
couragingly low agreement on some 

of these questions encourages a 
conservative attitude when inter- 
preting data drawn from interviews 
with mentally retarced respondents. 


Open-Ended Questions with Coded 
Response Categories 


Three open-ended questions poseå 
of community children and their parents 
required subsequent coding of responses 


_into categories. For example, responses 


to the questions "What ‘things would 
you really like to learn in school?" 
were assigned to such categories as 
academics, physical recreation, and 
self-care skills. For each of these 
questions, ten such response cate- 
gories were defined. Comparisons were 
conducted on a category-by-category 
basis, so that each category was coded 
either "mentioned" or "not mentioned" 
by client and parent. Responses were 
then compared in a 2X2 contingency 
table, computed by averaging across 
all 30 response categories (Table 7.5). 
Average agreement of these 30 categories 
was 77.9%. However, as can be seen, 
most of this agreement resulted because 
-both child and. parent did not men- 
tion a category. Only 6.8% of j 
client-parent comparisons yielded 
agreement because both client and 
parent mentioned a particuiar 
category. It is difficult to draw 
conclusions from these data, because 
agreement depends to a large extent 





Table 7.5: 


Mean Contingency Table for 30 Coded 


Response Categories to 3 Open-ended Questions 











Pes 
2. 












Responses of clients 





Did not mention 





Mentioned 





Response of Significant Others 


Did not mention 











Mentioned ` 


77.9% 


Agreement = 


7.10. 
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on the verbosity of respondents and 
the number of categories developed 
by the coders. 
that these open-ended. questions 
yielded very little information. 

It is also instructive to note that 
the four response categories which 
yielded the highest porportion of ` 
mention-mention agreements were the 
four categories lowest in overall 
child-parent agreement. This sug- 
gests that when clients and their 
significant others do offer infor- 
mation in response to open-ended 
questions, the responses they make 
frequently differ. In view of this 
lack of evidence for the validity of 
responses to open-ended questions, 
as well as the low responsiveness 
rates for these questions, it 

would appear that this format holds 
little promise in interviewing men- 
tally retarded persons. 


Conclusions 

Analysis of the open-ended ques- 
tions suggests two major conclusions. 
First, information drawn- from open- 
ended questions must be evaluated 
with caution. The low. agreement 
obtained on simple factual questions 
and the large variability in agree- 
ment across questions support this 
observation. Second, the open-ended 
question format is a relatively un- 
attractive one. These questions 
tend to pull very little information 
from mentally. retarded persons, and 
the. information obtained from open- 
ended questions has not been vali- 
dated by agreement with parents' 
responses. 


CONCLUSIONS 


` General Considerations 


These’ data regarding the validity 


of the résponses of mentally retarded 
persons are not encouraging. It is 
clear that information drawn from 
interviews with retarded persons 
must be evaluated critically. 


It is at least clear., 


Children's responses frequently fail. 
to correspond closely to the re- 
sponses of their parents. It is 
particularly notable that this lack 
of agreement exists for some simple 
questions regarding factual infor- 
matign which should be readily 
available to most persons. Second, 
it is apparent that different 
questions yield. responses. of dif- 
ferent validity. In some cases, 
particular topics seem to be af- 
fected by systematic response biases. 
Furthermore, conceptually more 
difficult questions appear to yield 
less valid responses. However, it 
seems difficult to predict in ad- 
vance which questions will yield. 
very low client-parent agreement. 
This fact is of vital importance 
because it implies that response 
validity can never be globally 
assumed; rather, each question must 
be interpreted conservatively since 
any individual question may be es- 
pecially subject to response effects. 
Unpredictable variability, coupled 
with the very low validity of many 
responses, requires that researchers 
who interview mentally retarded 
persons design interviews which deal 
very carefully with the issue of 
validity. , 


Format. Recommendations / 

The fact that major differences 
exist in client-parent agreement as 
a function of question format sug- 
gests that some formats are pref- 
erable to others. Yes-no questions 
appear especially unsuitable due to 
the effects of acquiescence. 


. Client-parent agreement frequently 


drops to below chance levels when 

a high proportion of parents re- 
spond "no" to these questions. 
Either-or questions, on the other 
hand, appear to have particular 
promise. Agreement for these ques- 
tions was relatively high and 
appeared to'be uninfluenced by 
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systematic biases. Multiple choice 
questions offering response aiter- 
natives which differ quantitatively 
are very unattractive; agreement on 
these questions was below chance 
level in this sample. In contrast, 
muitiple choice questions offering 
discrete response alternatives and 
dealing with basic factual informa- 
tion yielded the highest agreement 
rates of any format. Further ex- 
ploration of the potentials.of dis- 
crete multiple choice questions 
seems appropriate. On the basis 

of client-parent agreement the open- 
ended question format appears un- 
desirable. Average agreement was 
reasonably high when the question 
called for a Single factual answer, 
tut open-ended questions calling 
for an active listing of responses 
are probably useless. Moreover, 
the fact that many retarded persons 
cannot answer open-ended questions 
means that even when they yielded 
valid ansvec:: those answers come 
from only a limited segment of the 
population. In short, either-or 
questions or multiple choice ques- 
tions offering discrete alternatives 
would appear to have the most promise 
as methods of obtaining information 
fre’ .entally retarded respondents. 


ANALYSIS Or AGREEMENT USING 
COEFFICIENT KAPPA 
In the preceding @iscussion of 


agreement in the community children's 


sample, little mention was made of 
any standard against which agreement 
figures can be compared. We have 
simply compared agreement figures 
and applied an intuitive standard 
based on the anticipated ranges of 
agreement figures. At one extreme, 
if responding were completely random 
on the part of both clients and sig- 
nificant others, agreement for ques- 
tions with two response alternatives 
. would average 50%. At the other 
extreme, perfect agreement between - 


clients and their siqnificant others i 


would result in agreement of 100%. 
An agreement figure can be roughly 
evaluated according to where it 
falls on this continuum. Agreement 
lower than or little better than 
50% suggests low validity, while 
agreement approaching 100% .is-im~ 
pressive. (One must, of course, 
remember that agreement is a rough 
estimate of validity; e.g., for 

some questions, acquiescence in- 
flates agreement but actually raises 
questions about response validity.) 
Using this intuitive standard, the 
82.0% agreement for the hypothetical | 
questions depicted inTable 7.6 
appears to be reasonable evidence 
of response validity. 


However, as was done in Chapter 6 
on reliability, kappa coefficients 


' ean be calculated to compare ob- 


tained agreement to "change" agree- 
ments as defined by reference to 
the response: patterns (i.e., mar- 
ginals) of the two groups. Given 
the marginals depicted in Table 7. 6, 
the extent of agreement expected by 
chance for the hypothetical question 
is 82.0% (.9 x .9 + .l Xx .l). That 
is, given that 90% of clients and 
90% of significant others respond 
"yes" to this question, then 82.0% 
of. client-significant other pairs 
would agree if those pairings were 


' established by drawing randomly 


from the two populations. Thus, the 
actual agreement figure of 82.0%, 
impressive according to the intuitive 

standard, is actually no different. «. 
than the extent of agreement to be. 
expected by chance, and the kappa 
analysis makes us wary of making too 
much of the extent of agreement 
obtained. 


If we were concerned only with the 
aggregate level of analysis, refer- 
ence to agreement figures and 
associated kappas would not even be 
necessary. It would be adequate 
to look only at the marginals of 
contingency tables. If, as in the 





Table 7.6: 
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Hypothetical Yes-No Question 


r 








Clients 


10.0% 


Significant Others. 


Agreement = 82.0% 





example in Table 7.6, the percent- 
ages saying "yes" in each group - 

were identical or nearly identical, 
we would have a basis for concluding 
that the picture of the sample 
obtained is virtually the same 
whether one listens to clients or 

to their significant others. This 
- type of concurrence would, of course, 
be possible even if there were numer- 
ous discrepancies between individual 
clients and their significant others. 


If, on the other hand, one is con- 
cerned (as we have been) with. 
validity at the individual level of 
analysis, attention to paired 
responses is required. We have 
generally been content to use an- 
intuitive standard in judging agree- 
ment figures, for we feel.that 70 

to 80% or greater- agreement between 
clients and significant others is _ ! 
impressive. Moreover, it means that, 
we would have confidence, if we were 
to conduct correlational analyses , 
using.either client or significant 
other data in its place, that 
findings based on the two groups of 
informants would generally be 
similar. However, we can still ask 
whether a given agreement figure 
really represents agreement beyond 
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what one would obtain if client and 
significant other responses were 


‘paired randomly rather than correctly. 


It is for this purpose that we have 
chosen to use kappa selectively š 
simply to provide an additional 
perspective on consistency and agree- 
ment by chance; it is difficult for 
kappa to be large and significant 
when ;jthe marginals are lopsided 
simply because the probability of 
agreement by chance becomes large. 
Given an agreement figure 80 to 90%, 
we can still be satisified that 
clients and their significant others 
are saying much the same things, 

even if kappa is not significant. On 
the other hand, we cannot be fully 


_ satisfied by.low agreement, figures 


even if kappa is large, for we would 
not be confident using data from 
clients and significant thers as if 
they were interchangeabje. 


We can be most satisfied if agreement 
is high and kappa is high, for then 
we have reason to conclude that the 
axtent of agreement obtained is 
greater than what would be expected. 
if responses from the two groups 
were paired randomly. 
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AGREEMENT IN THE COMMUNITY CHILDREN'S chance according to the kappa 


SAMPLE BY THE KAPPA CRITERION criterion. All three of these 

questions were simple verbal 
Overall Agreement i questions having to do with 
Yvera = ee 


The use of kappa to analyze teaching skills. For example, 


agreement yields discouraging results. for the question "Is anybody at 
Of 88 client-parent comparisons for eee teaching you about cooking 
the community children's sample, only. now?" agreement was 71.1%, with 


11 resulted in agreement which kappa pie .42 (p less than .01, 
differed significantly from chance one-tailed). The low percentage 
(at the .05 level) according to this _of yes-no questions for which 
standard. agreement was significant by this 


criterion highlights the conclu- 
sion based on the earlier analysis: 
Agreement by Question Format yes-no questions do not elicit 
Multiple choice questions offering generally valid responses from 
discrete response alternatives emerge mentally retarded respondents. 


as the most desirable format when ~ However, it should he noted that 
kappa is the criterion. All six of the high proportions of "yes" 
these multiple choice questions responses on many of these ques- 
generated agreement significantly tions mean that a large kappa is 
different from chance. TO cite an difficult to obtain. 


example, for the question "Do you 
live in a duplex, a house, a trailer 
‚house, or an apartment building?" 
agreement was 81. 6%, with kappa = .44 
(p less than .05, one-tailed). It 
is unfortunate that these six ques- 
tions all dealt with either the 
client's means of transportation to 
school or with his/her dwelling place, 
as these topics were not adequately 
treated in other formats. Therefore, 
the possibility cannot clearly be 
rulea out that it is these topics, 
rather than the question format, which 
engender unusually valid responses. 
In short, analysis using kappa seems 
to recommend discrete multiple choice . 
questions even more strongly than did 
the previous analysis. However, none 
of the five multiple choice questions 
“using quantitative response alter- 
natives attained significance 
according to the kappa criterion. 
Thus the kappas reinforce the con- 
clusion that these types of multiple 
choice questions are not associated 
with high validity- of response. 


The remaining two comparisons for 
which kappa indicates agreement 
differing significantly from 
‘chance were open-ended questions. 
One of the four open-ended factual 
questions, “Most days, how do you 
get to school?" resulted in agree- 
ment of 85.4% (kappa = .68, Pp 
less than -01, one-tailed). ` The 
fact that the content of this 
question coincides with the content 
of three of the discrete multiple . 
choice questions may suggest that 
the especially impressive agree- 
ment for these questions is at 
least partly attributable to ques- 
tion content. Additionally, one 
of the 30. response categories for 
open-ended questions with coded 
responses generated agreement sig- 
nificantly different from chance 

_ by the kappa criterion. For the 
question "Do you play any sports? 
(if yes) What sports do you play?" 
the category "basketball" generated 
agreement of 75.0% with kappa = .43 
(p less than .01, one-tailed). 


Of the 32 yes-no questions in this - Given only two significant agree- 

sample, only. three generated agree- ‘ment figures for the 34 client- 

ment significantly different from parent comparisons for the open~ 
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ended questions, the Kappa analysis 
does not appear to challenge our 

` earlier conclusion that open-ended 
questions are not particularly 
appropriate for interviews with 
mentally retarded respondents. 
Again, however, the high pro- 
portions of "no mention" re- 
sponses among both children and 
parents make the probability of 
agreement by chance’ correspondingly 
high. 


One conclusion based on the intu- 
itive analysis of agreement is 
somewhat in ~pposition to the 
conclusions drawn from the kappa 
analysis. That is, none of the 
eleven either-or questions in this 
sample resulted in agreement 
differing significantly from chance 
‘by the kappa criterion. Because 
our earlier analysis suggested that 
either-or questions are, along with 
discrete multiple choice questions, 
‘the most desirable question type, 
this observation is problematic. 

We can again point out that the 
marginals for either-or questions 
are lopsided, elevating the standard 
for significant agreement. We can 
till conclude that agreement is 
substantial and that systematic 
response biases (e.g., selection 
of..the last option) are relatively 
weak, but we are forced to be more 
tentative about the merits of 
either-or questions. 


In i the analyses of agreement in the 
remaining samples which follow, we 
will not present kappa statistics. 
Our’ discussions will emphasize the 
sheer degrees of agreement obtained 
as well as the, systematic biases 
accounting for: client-significant. 
other disagreement. Conclusions 
based on kappa statistics are 
generally parallel from sample to 
sample. Repeatedly we find that 
simple questions yield similar re- 


v 


sponse breakdowns, often lopsided 
ones, among both clients and signif- 


icant others. Whether or not agree- 
, ment exceeds chance expectation is 


perhaps less important than whether 
or not clients and significant 
others are in substantial agreement. 


fa 


Additional Samples 

Extensive comparisons were made 
of the responses of..clients and 
attendants in both the institution- 
alized adult and institutionalized 
children samples. The vast majority 
of this information echoed -the con- 
clusions stated above. The following 
discussion will examine those in- 
stances in which data from the in- 
stitutionalized children or adults 
either contradicted or supplemented 
the conclusions drawn from the com- 


munity children sample. 


Overall Agreement 

A total ot 126 questions per- 
mitted comparison of responses given 
by institutionalized adults with 
responses given by their attendants; 
56 questions were asked of both in- 
stitutitionalized children and- their 
attendants. Average agreement 
figures for these samples and for 
the community „children are compared 
in Table 7.7. The overall agreement 
figures for the three samples are 
remarkably similar, suggesting that | 
whether informants are parents or 
attendants one can generally expect 
mentally retarded persons to agree 
with them slightly less than two- 
thirds of the time. Where questions 
of one type were asked of more than 
one sample, the similarity in agree- 
went figures is impressive. For the 
most part, the conclusions drawn from 
analysis of agreement in the commu- 
nity children's sample apply equally 
to the data from additional samples. 
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Table 7.7: Agreement by Question Type: . All Samples 


Ai , 
Community Inst. Inst. 







_ # of Questions 


Community Inst. Inst, 

Question type _ Children Adults — Children Children Adults Children 
Yes-No (words) 22 62 35 65.7 60.6 63.0 
Yes-No r SE 
(with pictures) 10 10 ⁄ 71.5 72.1 60.1. 
Either-Or = | . i ae : 
(words only) 7 4 / 2 65.6 71.4 72.3 | 

j I 
Either-0r f 
(pictures) 4 i 58.1 


Multiple choice l 
' (discrete) 6 7219 ; 


Multiple choice 
| (quantitative) 5 | 26 l. 24.0 a0 25.6 





Open-Ended (factual) |- 

Open-Ended (numerical) . ; l o | 7.6 

Open-Ended (coded 7 

categories) » 3 19 5 71.9 79.9 72.0 
a 6l 126 + 56 645 593 62.8 
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ADDITIONAL INFORMATION FROM THE 
INSTITUTICNAL SAMPLES 


~Yes=No Questions in the Institution- 


alized Adult Sample 

In the’ discussion of the commu- 
nity childrën' s sample, it was 
suggested that a group of questions 
to which 50% ò the responses of 
significant others were "yes" would 
give an improved estimate ‘of re- 
sponse validity. Nine simple yes-no 
questions for the \institutionalized 
adult sample provide just such an 
For 22 simple yes-no 
questions in the community children 
sample, 68.8% of parents' responses 
were "yes," and agreement was 65.7%. 
For the 44 simple yes-no questions 
asked of institutionalized adults, 
47.7% of attendants' responses were 
"yes," and agreement dropped to 
57.7%. It is apparent that when 
agreement is not inflated by 
acquiescence, agreement for yes~no 


“questions is only slightly better 


than the 50% that would be expected 
if responding by both clients and 
their significant others were com- 
pletely random. 


Rules Questions : 

_ Eight yes-no questions dealing 
with rules ("Are you allowed to stay 
up late at night?") were posed of 
institutionalized adults and yielded 
a pattern of responses quite dif- 
ferent from that typifying yes-no 
questions in general. For four 
questions, the wording was "Is it 
against the rules...?" and for the 
other four, the wording was "Are 
you allowed...?" All questions 
dealt with behaviors that are 
These ques- 
tins were paired so that the two 
sets of four dealt with precisely 
the same content but were worded 
oppositely. Agreement for the "Is 
it against the rules...?" questions 
was only 53.4%, with 35.3% of the 


i 


comparisons disagreements in which 


the clients respc^>ðed "no" and the 
attendant "yes." Agreement for the 
"Are you allowed...?" questions 
averaged 79.9%, with 70.3% of com- 
parisons agreements in which both 
respondents answered "no." It. 
appears that questions dealing with 
obvious rules and phrased "Are. you 
allowed...?" are unusually conducive 
to valid responding. By contrast, 
respondents appear to have misunder- 
stood the "Is it against the 
rules...?" questions and may have 
responded "no" thinking that they 
were indicating that the behavior 
was not allowed. Even more inter- 
estingly, agreement for these |ques- 
tions was clearly not influenced by 
acquiescence.since most agreements 
consisted of "no" responses from 
poth client and attendant. Retarded 
persons do not acquiesce o any 
yes-no questions. (In fact, here 
the correlation between proportion 
of attendants responding "yes" and 
agreement was -.75, a direct 
reversal of the more usual trend.) 
These questions, in combination 
with the skills questions discussed 
earlier, suggest that the retarded 
are most likely to acquiesce when 
that is the socially desirable re- 
sponse but do not when "no" appears 
to be socially desirable. Clients 
appeared to have a clear sense that 
the activities asked about were 
socially unacceptable. 


Last-Option Position Bias in 
Either-Gr Questions 

Within the either-or questions 
asked of institutionalized adults, 
there was evidence for a weak 
position bias on the part of mentally 
retarded respondents, who showed a 
slight preference for the last- 
mentioned alternative. Asked to 
specify whether they are usually 
happy or sad, 91.2% of clients re- 
sponded "happy" when that was the 
last-mentioned alternative, as 
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opposed to 84.3% when "sad" was 
mentioned last. When asked if they 
are usally alone or with others, 

_ 64.5% of respondents said "with 
others" when that alternative was 
mentioned last, as, opposed to 

51.5%: when " alone” was mentioned 
last. This response set tends to 
elevate agreement slightly when 

the laat-mentioned alternative is 
the response preferred by attendants. 
However, this effect is very small ` 
by comparison with the effects of 
acquiescence. Agreement for either- 
oy questions in the adult institu- 
tional sample supported our earlier 
conclusion that the validity ‘of 
responses to either-or questions 
exceeds the validity of .responses 

to yes-no questions. 


Open-ended Questions with Preset 
Categories 

Four questions asked of insti- 
tutionalized adults and their 
attendants were open-ended questions 
for which responses were coded into 
one of two preset categories. For 
example, all responses to the 
question "Who decides how you spend 
your money?" were coded into’ the 
categories “somebody else" or 
"client." Average agreement for 
these questions was 81.6%, ranging 
from 70.6%-to 92.6%. 
open-ended question—"How many 
people sleep in your bedroom?" re- 
quired numerical responses and re- 
sulted in agreement of only 7.6%. 
Interviewees were totally unable, to 
provide valid answers to this ques- . 
tion.) The average agreement. figure 
of 81.6% for the four questions far 
exceeds that of 64.3% for the open- 
‘ended factual questions in the 
community children's sample. It 
appears that open-ended questions -~ 
soliciting simple information and 
using preset categories may be 
more promising than was concluded 
on the basis of interviews of 
community children. In fact, the 
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unusually high agreement for these 
questions casts this format in a 
very favorable light, although it 


‘ is still the case that many inter- 


viewees cannot answer such questions. 


Conclusions 

In short, the data from addi- 
tional samples modify the conclu- 
sions drawn from the community 
children’ sattiple in a few ways. 
First, acquiescence does not occur 
to all questions. The "rules" 
questions showed that under specific 
circumstances, this response bias 
can be outweighed by other factors 
(possibly a social desirability re- 
sponse set). Second, either-or 
questions in some cases engender a 
last option position bias, which to 
some degree reduces the validity of 
responses to these questions. 
Third, open-ended questions with 
preset response categories may be 
somewhat more useful than was con- 
cluded on the basis of responses 
of community children. : In general 
the conclusions drawn from the 
community children sample are 
supported by these further analyses. 


Correct-Incorrect Questions as a 


Measure of Response Validity 


ALL QUESTIONS 

Although client~significant 
other agreement served as our basic. 
measure of response validity, a 
number of questions posed of mentally 
retarded respondents had known 
correct answers, allowing for an- 
other type of ‘analysis of validity. 
Of nineteen such questions, the re~ 
sponses to twelve were coded as 
either incorrect, partially correct, 
or correct, while responses to the 
remaining seven were coded either. 
incorrect or correct. For the 
entire group of twenty, questions, 
77.1% of clients' responses were 
correct or partially correct. For 
the twelve incorrect-partially~correct 
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questions, an average of 18.9% of reveal that about three-quarters 


responses were incorrect, 19.9% ` of clients wno can answer the 

were partially correct, and 61.2% question are able to give their full 
were correct. For the seven ques- names correctly, and another quarter 
tions coded simply incorrect or are able to state either their first 
correct, an average of 29.7% of or last name. Only one of the 108 
responses were incorrect, and an clients who responded appropriately 
average of 70.3% of responses were to this question was unable to give 
correct. Because such average at least a partially correct answer. 
figures are difficult to conceptu- " In one sènse, this seems sbmewhat 
alize, these questions are dis- encouraging. However, it is impor-: 
cussed on a more detailed level ~ tant to remember that many \lients 
below. 


were unable to respond to this 
question. In fact, when thdse 
QUESTIONS ASKED OF ALL SAMPLES who were asked but were unrespon- 


Hwo:-epencsended martone wate sive to this question are considered, 
asked in Peat oun of respon- only 64.5% of all communtEy ehis 
dents in the three major samples dren, 40.2% of institutionalized 
included in our study: community, adults, ana 50.0% Gf instituton 
children, institutional adults, and alieed Changeen who were asked the 
‘institutional children. The first meseron kiea able to COrrectiy 

_ question, ."What is your full name?" anes meee PO E 
yielded the results depicted in Since ela nen = rl a sees ene 
Table 7.8. Answers were coded as most basic piece of eaan i 
partially correct if the respondent hat anyone ans! Constderaeson 

‘made an obvious attempt to say pf this question goula perhaps 
either the first or last name, and > eee as s sobering Foundation for 
correct if both the first and last . Qur thinking regarding interview- 
‘names were stated correctly. In ing retarded persons. If only 
general, the data in Table 7.8 around half of mentally retarded 


persons can correctly state their 





-Table 7.8: Percentage of Correct Responses to "What is your full name?" 














Partially 
Correct 











Incorrect Correct 


Community children 
Institutionalized adults 


Institutionalized children 
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full names in response to a direct 
question, then the implicatica of 
the previous portion of this chap- 
ter, that the validity of the re- 
sponses of mentally retarded indi- 
viduals is frequently minimal and 
always in question, is no way’ sur- 
prising. The most optimistic in- 
terpretation of this information is 
that when retarded persons do 
respond appropriately to very simple 
questions, their responses can be 
expected to be at least partially 
correct. 


Additionally, the question "How do 
you spell your name?" was asked of 
the three samples. In response, 
74.5% of community children, 71.4% 
of institutionalized adults, and 
82.8% of institutionalized children 
have responses which were at least 
partially correct. This information 
perhaps pertains more to our re- ` 
spondents' language skills than to 
response validity per se. 


QUESTIONS ASKED OF TWO SAMPLES 

Two additional questions were 
included in the interviews with 
both community children and institu- 
tionalized adults. The first of 
these questions, ."Please write your 
name for me on this paper," was 
again more a test of language skills 
than of respcnse validity per se. 


SS 


Table 7.9: 
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Commur:ity children 

eee a a a 
Institutionalized 
adults 


tutionalized adults were at least 


Incorrect 


Among community children, 77.8% of 
respondents gave correct or partially 
correct answers, and 60.0% of insti- 


partially correct. i / 


The second question was phrased 
slightly differently for the two i 
samples. "What is your address at K 
home?" was the wording for the comy 
munity children, as opposed to "what 
is your address here?" asked of ; 
institutionalized adults. Responses 
of institutionalized clients were 
coded correct if they mentioned ; 
both the institution and the, /oity, and 
partially correct if they ‘mentioned 
one but not the other. Responses 

of community children were coded 
correct if they gave a mailing or | 
street address, and partially correct 
if they gave a street name without a 
house number. The fraction of 

correct responses, depicted in Table 
7.9, gives us another solid insight 
regarding the issue of response 
validity. This question involves 
information which should be readily 
available i> most persons, but it 

was apparently considerably more 
difficult than the "name" questions. 


ADDITIONAL QUESTIONS 

A number of additional factual 
questions were asked of only one 
sample. Three open-ended questions, 


ee 


Percentage of Respondents Correctly Stating Their Addresses 






Partially 
Correct 





k 
Correct 








coded incorrect or correct, were 
asked of institutionalized adults. 
These three questions, “What month, 
is your birthday?" "What day is 

your birthday?" and "What year 

were you born?" resulted in 80.6%, 
77.8%, and 64.7% correct responses, 
respectively. Again, those clients 
who were able to respond appropri- 
ately generally gave accurate infor- 
mation. However, if we take intq 
account unresponsive persons, only 
28.2% of those clients asked the 
question were able to state their 
day of birth, 48.6% correctly named 
the month in which they were born, 
and only 20.1% correctly named their 
birth year. In short, méntally re- 
tarded respondents are often unable 
to correctly answer even very simple 
questions, but this inability” usually 


takes the form of a failure to respond. 


appropriately, so that those who do 
respond appropriately are likely to 
also respond correctly. 


Four yes-no' questions having known 
correct answers, all involving the. . 
value of money, were included in the 
interview of institutionalized adults. 
These questions were included to 
assess subjects' understandings of 
money concepts. For the questions 
"Could you buy some candy with $100?" 
"Could you buy: a new bicycle with 
$1.00?" "Could you buy a new TV with 
$10.00?" and "Could you buy a record 
album with $10.00?" the fraction of 
correct responses was 38.9%, 60.5%, 
27.0%, and 92.3%, respectively. It 
is especially interesting to note that 
the questions for which the correct 
answer was "yes" had by far the 
highest fraction of correct responses, 
another instance, of the effect of 
acquiescence. This factor makes it 
difficult to evaluate the validity of 
responses to these questions. The- 
fact that 50% of respondents would be 
expected to answer these questions 
correctly by chance and only 27.0% 
correctly answered the TV question 
casts serious doubt on whether re- 


tarded persons can be gxpected to, 
answer questions 7 the value of 


money. 

Finally, two open-ended questions 

were asked only øf the institutional 
children. In ņésponse to the ques- 
tion "What is/your birthday?" 66,7% 

of children /gave correct responses, 
27.3% gave’ partially correct responses 
and 6.1% gave incorrect responses. 

For the question "What kind of place 
is this?" 61.9% responded correctly 
(by referring to the institution) and 
38.1% incorrectly. Once again; our 
clients showed a clear but limited 
ability to respond correctly to simple 
anestions, if they can respond at all. 





CONCLUSIONS 

As with the agreement data, the 
results of these factual questions 
must leave us pessimistic but not 
without hope. Mentally retarded 


' individuals do tend to respond cor- 


rectly to these simple, straight- 
‘forward factual questions. However, 
a discouragingly high proportion of 
respondents respond incorrectly when 
very fundamental information is re- 
quested of them: name, birth date, 
address, and so on. Furthermore, 
when responsiveness data are con- 
sidered in tandem with information 
about validity, it is clear that only 
a small proportion of retarded persons 
can correctly answer even these "easy" 
questions. Examples of some of the 
difficulties we encountered are in- 
cluded in Sample 7.1. .These correct- 
incorrect: questions provide a succinct 
summary of all our data regarding 
validity, in that they demonstrate 
that mentally retarded persons can, 
under some circumstances, provide 
valid information about themselves, 
but that the validity of the infor- 
mation they can provide can never be 
taken for granted. 


“7.21 


139 


- Sample 7.1 


A Sampler of Problems in Obtaining Basic Factual Information 


Is Q: What is your full name? 
A: (Lifted her foot and pointed to the bottom of ‘shoe, where her : name 
was written) 
2. Q: What is your birthday? 
A: I don't know. — They got it in the records. I'm 15. I think 


it's February. 


3. Q: What is your name? 
A: My name is Ronnie Peters. 
Q: How do you spell your name? 
A: RONKLTUWXYZ 
4. Q: What year were you born? 
A: Little Rock. 
5. Q: Do you get SSI? _ 
A: What? 
Q: Do you get SSI?- 
A: SSI? Spell it!! N 
6. Q:` What is your address\ here? 
A: 35944234 © sA 
7. Q: What is your full name? 
A: Huh? I don't got no name. = 
Q: Do you get SSI?- 
A: I gots my bus ticket. 
Q: (repeated) 
A: I guess. 
Q: Do you have a family? 


No except my mama and daddy. That's all I have. 


7 


Q: (repeated) 
A 


7 No. 
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Agreement With Parents and 
Attendants as an Individual 


Characteristic 

To provide another perspective 
on client-significant other agreement, 
we focused on 13 yes-no questions that 


were asked in precisely the same form 


to institution children, commun ity 
children, and institution adults. 
Nine of the questions were about par- 
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ticipation in various chores and were 
accompanied by pictures, while the 
remaining four were simple verbal 
yes-no questions. An agreement score 
was calculated for each subject who 
had responses paired with significant 
other responses on at least half of 
the set of 13 questions. The agree- 
ment score could range from 0 ic 100%. 
We sought to determine whether agree- 


‘ment differed from sample to sample 


S 


„Institution Children (N=45) 
er 
"community Children (N55) 


Institution ‘Adults (N=38) 
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Figure 7.2 Agreement with Parent By !Q Group for I3 Yes~No Questions Used 


In Three Samples 
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. the three samples. 


when the same questions were at issue 
and to explore individual differences 
in the tendency to agree with parents 
or attendants. 


Figure 7.2 presents the mean agreement 
scores for severely ,. moderately, and 
mildly retarded subjects in each of 
As the figure in- 
dicates, there was some tendency for 
agreement with significant other to 
increase as IQ increased. A 3 X 3 
analysis of variance indicated that 
there was a significant difference 
between IQ groups (F (2, 129) = 3.62, 
p = .03), but” ‘that. the three samples 
were not significantly different 
from each other and that IQ group 
and sample did not interact. Over- 
all, severely retarded persons 
agreed with their significant others 
70.5% of the time, moderately re- 
tarded persons 74.4% of the time, 
and mildly retarded persons 79.7% of 
the time on these questions. 
up t-tests with the three samples 
collapsed inå-cated that severely re- 
tarded persons were significantly 
less likely to agree with their 
significant others than were mildly 
retarded persons, t (135) = 2.56,p = 
.012. Although the interaction he- 
tween IQ group and sample was not 
significant, there were some varia> 
tions in findings from sample to 
sample. Among institutionalized 
children, the severely retarded 
group was definitely deficient in 
comparison to the mildly retarded 
group, t (42) = 2.66, p= .02. In 
the community children's sample, the 
difference between severely retarded 
and mildly retarded respondents 
approached significance (p = .056). 


‘Interestingly, there were no signifi- 


cant IQ group differences in the 
adult institution sample, and in 
fact agreement scores actually de- 
creased slightly as IQ increased. 
Altogether, then, this analysis 
revealed a tendency for agreement 
with significant other to be a func- 


Follow- 


tion of level of retardation, 
although it was certainly not a 
strong pattern even though it was 
statistically significant. Moreover, 
level of agreement did not differ 


-greatly from sample to sample;—despite 


the fact that the parents of children 


. living in the community wight be ex- 


pected to be more knowledgeable in- 
formants than institution attendants 
responsible for many residents. Of 
course, it must be borne in mind that 
this analysis could only be conducted 
on yes-no questions, which, as dis- 
cussed previously, often appear to 
yield biased answers which may inflate 


estimates of validity as indicated by 


agreement with significant others. 
Nonetheless, for such questions one 


can apparently expect agreement with r~ 


significant others, whether tHey are 
parents or institution staff, to 
average about 75% and to be weakly 
but positively related to the 
respondent's IQ... 


To show more concretely the levels of 
agreement which were obtained, Tables 
7.11, 7.12, and 7.13 report for 

each sample the percentage of 

clients and significant others 

saying "yes" to each question and 

the percentage of cases in which 
answers agreed. As noted in Table 
7.12, there was only one instance 

in which the kappa statistic indi- 
cated agreement beyond the level 
expected by chance (cleaning floors 
in the adult institution sample). 

At the same time, the percentages 
agreeing are exceptionally high on 
Many other questions, and it must 

be borne in mind that extremely / 
lopsided distributions in which 
virtually everyone says "yes" miti- 
gate against obtaining statistically 
significant agreement. The tables 
add still another perspective, for 

in some cases, despite relatively 

low agreement figures, the proportions 
of clients and significant others 
saying "yes" are very similar. For 





Table 7.11: Agreement in Children's Institution Sample , 


a Se 





ra a j i 
Yes according | Percent 
to 50/ 


$ Yes according 
to clients 








Activity 
Watch TV 
Listen radio, record player 
Go to church 
Count noney (knows how) 
Set table 


Do dishes 





‘clean the floor 
Dust bata 
Make bed | 
Pick up stuff 
— Cook on the stove 
` Make sandwiches (food) 


Take. out trash 


Activity to clients Kappa 
' Watch y \ . 000 
Listen radio, record player | „000 
Go to church 2 
Comt ny (knows how) -.074 
Set table 1263 
Do dishes ky 
Clean the floor 069 
Dust furniture 017 
Nake bed 077 
Pick up stuff ~.057 
Cook on the stove ~.074 
Make sandwiches (food | „090 
ake out trash ; : 157 
A woe 


\ 
Table 7,12) Agreement in Comunity Children's Sample 


, , bg 
+ Yes according | % Yes according] Percent 
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Table 7.13: Agreement in Adult Institution Sample 


$ Mi 





% Yes according | % Yes according) Percent 





Activity | | to clients | Agreeing 

Watch TV 97,5. 

Listen radio, record player. | _ 89.7 

Go to church 94,7 

Count money (knows how) 11.9 

set table 19.5 

Do dishes | | 78.9 
“Clean the floot 8.9 
__Dust_ furniture oT 

Make bed | | 89.2 

Pick up stuff | 83.8 

Cook on the stove 67.6 

Make sandwiches (food) | | 75.0 

ake out trash 86.5 





*2=1.7, significant at the 5 level, 
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example, while institutionalized 
adults and their attendants were in 
agreement only 61.1% of the time 


-about the. client's involvement in 


setting the table, the 79.5% of the 
clients indicating involvement is 


not very different from the 76.9% of 


the attendants indicating involve- ; 
ment. “In such cases, the implications 


“o£ disagreement between clients and 


significant others are not severe if 
one is only interested in aggregate 
figures, for despite disagreement at 


_ the individual level, pictures of 


the group emerging from the two 
groups of informants are much alike. 
In other cases, however, it is clear 


' that one does not obtain the same 


information from the two groups of 
informants. The most markedly dif- 
ferent pictures of the group were 
obtained on the questions about 
whether the client knows how to count 
morey and cooks on the stove. Ac- 
cording to the clients themselves, 
for example, 88.5% of the institution 
children, 80.8% of the community — 
children, and 71.9% of the institu- 
tion adults know how to count money. 
The corresponding figures based on 
significant other responses were 


only 42.3%, 61.5%, and 31.3%, respec- 


tively. Similarly, according to 
clients 61.0% of the institution 
children, 65.5% of the community 
children, and 67.6% of the institu- 
tion adults cook on the stove. The | 
quite different figures provided by 
significant others were 7.3%, 53.7%, 
and 8.8%, respectively. In both 
cases, clients, by saying "yes" 
frequently, are apparently over- 
stating their involvement. These 
figures reinforce once more the 
variations in agreement associated , 
with individual content areas and 
the value of interpreting agreement 
from more than one perspective. 


CORRELATES [OF AGREEMENT 
Given the fact that acquiescence 
on yes-no questions has emerged as 


a problem in interviewing the men- . 
tally retarded, we attempted to look 
at agreement with significant others 
in relationship to acquiescence. 

We correlated the agreement score 
described above with a dichotomous 
acquiescence score which simply in- 
dicated whether or not a client 
responded "yes" to both the question, 
"Are you usually happy?” and the ques- 
tion, "Are you usually sad?" To. 
further explore the nature of agree~ 
ment with significant other as an 
individual characteristic, we also 
correlated the agreement measure 
with IQ, sex, anda responsiveness 
score developed on the basis of 22 
items used in all three samples. 
Table 7.14 presents these correla- 
tions for each of three samples. 


Whe: does this table reveal about 
agrsement? First, as was indicated 
‘in the previous discussion, higher . 
IQ respondents were more likely to 
agree with their significant others 
except in the sample of institution- 
alized adults. „Moreover, those 
clients who were able to answer a 
greater proportion of questions 
appropriately were generally more in 
agreement with their significant 


_ others, although again this relation- 


ship does not hold in the adult 
sample. (It should be noted that 
agreement scores were calculated 
only if a client had been able to 
respond to at least half the ques- 
tions in the set of 13, and the 
correlation coefficients are based 
only on those subjects who had 
scores.) In the children's institu- 
tion sample only, males were more ` 
likely to agree with significant 
others than were. females, while in 
the community children's sample this 
relationship was reversed, in direc- 
tion, and in the adult Sample non- 
existent, leaving us uncertain about 
the relationship between sex and 
agreement. Finally, in the children's 
institution sample, respondents who 
acquiesced were significantly less 
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Table 7.14: Correlations of Measures In Three Samples 





























IQ Inst. child 
Comm. child 
Instit. adult 
Sex 


Agreement 


Responsiveness 


*Significant at the .05 level 
**Significant at the .01 level 


a a 


Response | Acquiescence 








Note: In each, cell of the matrix, correlations in order moving downward are for the 
institution children, community, and institution adult samples, respectively. Sex 


is coded male = 1, female = 2. 


likely to agree with their signifi- 
cant others, but this negative rela- 
tionship was weak in the other 
samples. The relationship between 
acquiescence and agreement is diffi- 
cult to unravel. Lower IQ clients, 
while they tend to disagree with 
informants, also tend to acquiesce 
a great deal, and as we have noted 
previously acquiescence may inflate 
agreement if informants often say 
"yes," Curiously, relationships 
were consistently stronger in pre- 


-4 dicted directions in the children's 
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institution sample than in other 
samples. Generally, trends in the 
same direction were evident among 
community children, but none of the 
relationships. held among institu- 
tionalized adults. As this table 
also indicates, IQ was quite consis~ 
tently related to responsiveness in 
a positive direction and to 
acquiescence in a negative direction 
(i.e., higher IQ subjects were more 
responsive and less likely to acqui- 
esce), and similarly, highly respon- 
sive subjects were less likely to 
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acquiesce than were less responsive 
subjects. 


Conclusions . 
This chapter has examined the 
validity of answers given by mentally 
retarded interviewees in two ways; by 
‘comparing their answers with parallel 

responses given by attendants or 
parents, and by comparing their 
answers with known fact. As this 
analysis has indicated, establishing 
whether or not answers are valid, 
especially through the first approach, 
is problematic. We had no alternative 
most of the time but to look at each 
qüestion individually and then try to 
detect patterns in the data for sets 
of questions. 


What can we conclude overall? We can 
conclude, first of all, that the in- 
formation provided by retarded persons 
and the information provided by non- 
retarded informants about retarded 
persons is not necessarily the same. 
Indeed, in many cases one would obtain 
very different pictures of the needs, 
attitudes, and circumstances of re- 
tarded citizens from the two types of 
respondents. Our data suggest that 
their answers match approximately 
two-thirds of the time overail. The 
implications of this basic finding 
are serious. Mental retardation 
researchers (and practitioners as 
well, in their informal information- 
gathering efforts) have not been 
sensitive to this phenomenon. They” 
have often relied on one or other 
source of information, assuming,. but 
only. assuming, that data from the 
two sources would be interchange- 
able. To give an example, consider 
the literature on deinstitutionali- 

jon, which includes studies 

ed on client interviews and 
studies based on interviews with 
institution staff, houseparents, 
and other nonretarded informants. 
Studies based on different sources 
of data can be compared, but in view 
of the analyses presented here, one 


must recognize that findings may `. 
‘differ from study to study because 
data were collected from different 
informants. l 


At the outset, we warned that one 
canst always assume that retarded 
pers Ms are wrong when they disagree 
with their parents or attendants. 

We did indeed fiñd instances in 
which retarded persons and their 
significant others simply had 
legitimate differences of opinion. 
However, there were other instances 
in which we would point to response 
bias on the part of retarded persons 
as the cause of disagreement. It is 
these latter cases that allow us to 
conclude that retarded persons are 
sometimes "wrong" in the sense that 
their answers do not reflect the 
state of things as much as they 
reflect the operation of response 
sets. The clearest case of all is 
the threat to validity of response 
to yes-no questions represented 

by acquiescence. As a general rule, 
retarded persons and their parents 
or attendants agreed on yes-no 
questions only to the extent that 
parents and attendants happened to 
say "yes" frequently, thus matching 
the frequent "yes" responses of 
clients. This phenomenon raised 
serious questions about the very 

use Of agreement figures as a measure 
of validity of response. In short, 
we had to take into account the 

fact that high agreement was not 
necessarily evidence of validity, 
not when we could identify other 


“factors such as acquiescence that 


were artificially inflating agree- 
ment figures. 


We encountered similar problems in 
analyzing other types of questions. 
For examp].cs, since very few 
responses were obtained to open- 
ended questions calling for enumera- 


-tions, hich agreement figures for 


categories of response to sich ques- 
tions generally said little more than 
that clients and nonretarded infor- 
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mants frequently did not mention 
something. Thus it must be recognized 
that many -of the average agreement 
figures presented in this chapter are 
misleading indicators of validity of 
response. While we have generally 
found that retarded people and their 
significant others agree about two- 
thirds of the time, that 

does not mean that responses are 
valid about two-thirds of the time. 


On the positive side, we did find 
either-or questions to have respect- 
ably high agreement figures without 
the pronounced response bias that 
characterized yes-no questions. We 
also found high validity of response 
to multiple choice questions about 
` basic factual information such as 
type cf residential setting where 
respondents chose among discrete 
response alternatives (compared with 
extremely low validity for multiple 
choice questions with quantitative 
response options). And finally we 
found relatively high validity for 
factual open-ended questions that. 
required a single answer rather than 
requiring enumerations of responses. 
At the same time, none of the agree~ 
ment figures was high enough that 
‘we can be completely confident in the 
information obtained from retarded 
persons. We must still conclude that 
their answers cannot be assumed to \ 
be valid unless there is evidence 
to support that assumption. The 
inability of many retarded persons 
to provide basic factual information 
such as full name, birth date, and 
address simply reinforced that con- 
clusion. i 


-problem for yes-no questions. 


Is it possible to predict which 
retarded persons are likely to pro- 
vide valid answers? We attempted 

to answer this question by calcu- 
lating agreement scores for indi- 
viduals reflecting their tendency 

to agree with their significant 
others on a set of yes-no questions 
that were used in 111 three major 
samples. We found that children 
living in the community were, sur- 
prisingly, no more likely to agree 
with ther parents than institution- 
alized persons were to agree with 
their attendances, even though we 

had expected parents and children 

to be more likely to share a common 
set of facts and perspectives. 

While we found that higher IQ persons 
were more likely than lower IQ persons 
to agree with significant others, the 
relationship was not really strong 
enough that one could use it to pre- 
dict in advance whose answers will 
be valid. This matter needs to be. 
explored further, since our findings, 
which were restricted to yes-no 
questions, might not generalize to 
other types of questions, particu- 
larly in view of the acquiescence 

) Until 
further research is done, one must 

be wary of responses given by any 
retarded person, especially on the ` 
kinds of questions that we have found 
to elicit biased responses. 


\ 
In the next chapter, we will pursue 


the topics of response bias and | 
response validity through systematic 
comparisons of alternative ques- 
tioning approaches. 
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Chapter 8 


WHAT INTERVIEWING APPROACHES 
ARE MOST USEFUL? 


A major focus cf this report has 


@ 
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been evaluation of different kinds 
of questions according to the cri- 


teria of responsiveness, reliability 


and validity as.indicated by agree- 
ment with parents and attendants. 
To facilitate this evaluation, all 
of tne interview ‘schedules used in 
the various studies included spe- 


cific comparisons of alternative 
ways of soliciting-the.same—infor=........ 


mation, head-to-head competitions 
among different formats and phras- 
ings applied to the same c:ntent 
area. These comparisons were done 
specifically to identify question- 
ing techniques that optimize ‘re- 
sponsiveness, reliability, and 
validity. 


To illustrate the approach used, 
consider the comparison summarized 
in Sample 8.... In comparing two 


. questions, in this case a yes-no 


question and a parallel open-ended 
question, we first apply a criter- 
ion of responsiveness. We would 
generally prefer a question that 


can be ai.zwered by many persons to 
one that relatively few can answer. 
In the example, the institutional- 
ized adults studied found it con- 
siderably easier to answer the yes- 
no question than to answer the 
open-ended question, a finding 
which is entirely consistent with 


our analyses of responsiveness as 


a function of ene format (see 


Chapter. BY... 


A second criterion centers on re- 
liability in the sense of consis- 
tency between answers to the two 
alternative questions. The con- 
sistency of responses to these two 
questions was relatively low. 

While that alone does not’ give us a. 
basis for preferring one question 


over the other; it is also apparent ~-- 


that there is a systematic trend 


in the inconsistency. Namely, clients 


are highly likely to say, "Yes, I 
decide," in response to the yes-no 


question while they are mentioning . 


someone other than themselves-—in 
response to the open-ended question. 
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Sample 8.1 


COMPARISON OF ALTERNATIVE FORMATS 
Yes-no versus Open-ended 
(Do you decide what chores you do? versus Who decides 


what chores you do? -- Adult Institution Sample) 


“1. Which Question Yields Higher Responsiveness? 
Percent Responding Appropriately: Yes-no 54.4% 
| Open-ended 34.5% 
2. To What Extent Are Anwas Consistent? 
With answers coded client decides versus someone else 
decides, percent consistent = 56.3%. All of the rest 
(43.7%) are instances of "Yes, I decide" on yes-no and 
mention of someone else on open-ended--i.e., acquiescence. 
3. To What Extent Do Clients And Significant Other Agree? 
For yes-no, 34.6% acze. Remaining 65.4% are cases in 
which client says, "Yes, I decide," while attendant men- 
tions someone else in response to open-ended question: _ 
For open-ended, 88.2% agree. 
CONCLUS ON: While more subjects can answer the yes-no question, 
acquiescence invalidates response. Open-ended -ques- 


tion is preferred on grounds of validity. 
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This suggests that acquiescence in 
response to the yes-no question is 
operating and is inflating the 
number of respondents who claim 
they make their own decisions. 


Finally, we can apply a criterion of 
agreement with significant others 
(parents or attendants) in decid- 
ing which question is the best 
approach to use. Due to the acqui- 
escence detected in the consistency 
analysis, agreement with atter.dants 
is lower for the yes-no question 
than for the ‘open-ended alternative. 
On the basis of the agreement 
criterion, the open-ended format 

is therefore preferable. 


As is true in many of the analyses 
to follow, the application of 

these three criteria often involves 
trade-offs. While the yes-no ques- 
tion is to'be preferred on grounds 
of responsiveness, it is clearly 
deficient on grounds of validity 
because of the strong influence 

of acquiescence. Since there is 

no point in getting answers if 

they are not trustworthy as valid 
answers, we must ultimately prefer 
the open-ended question in this 
case. Yet neither alternative is 
entirely attractive and use of the 
open-ended question will ultimately 
mean that data will be obtainable 


of the population under study. 


The comparisons which follow are 
often more complex than the ex- 
ample, but they follow the same 
general logic. While it will prove 
impossible on ‘some occasions to 
clearly establish onexalternative 
as superior to another, we will. 
constantly be trying to determine 
which ways of asking questions 
Siper to work best in hopes of 
“way” aghing empirically bast , 
_.ucomendations for persons pitt 
ested : : interviewing retarded 


The chapter will consider in turn 
three major topics in question de- 
sign. First, we will report on 


. several comparisons of questions 


whose wording differed but whose 
formats were the same, asking the 
question: To what extent are the 
responses given influenced by the 
manner in which a question is 
worded? Secondly, we will briefly 
consider a procedural issue relevant 
to the ‘use of open-ended questions: 
What are the relative merits of 
simply asking the question as op- 
posed to asking the question and 
then persistently probing with 
"What else?" in hopes: of eliciting 
more information? This question 
is significant given our difficul- 
ties eliciting much information in 
response to open-ended questions. 


Finally, we address perhaps the 
major question underlying our 
studies: Which question formats 
appear to be associated with the 
greatest responsiveness, relia- 
bility and validity of response? 

We began our studies predicting 

that we would have difficulty get- 
ting answers to open-ended questions 
and that more structured questioning 
approaches might be more effective. 
In view of the limited verbal skills 
of many retarded persons, we also 
sought to determine whether the | 


use of pictures in interviewing) 77" 


might facilitate answering. ‘We(will 
begin describing our comparisons of 


-alternative formats by considering 
.the use of the multiple choice for~ 


mat in gathering information about | 
activities. Next, we will consider 
yes-no questions, comparing them 
with open-ended and either-or for- 
mats, as well as assessing the ef- 
fects of pictures as a Means of 
clarifying yes-no questions. Finally, 
we will analyze responses 

to either-or questions and explore 


_the potential value of using pic- 





individuals for research or program 
planning purposes. 


tures as response alternatives in 
this format. 
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Most comparisons of alternative. 
formats were conducted a number 

of times with different content 
areas. Because of this repetition, 
it would be unwieldy to present all 
data comparing two formats. Thus, 
we will follow the general strategy 
of selecting one comparison con- 
sidered to be representative of 

the others, and presenting that 
comparison in some detail. Other 
comparisons of the same type will 
be cited briefly tu supplement in- 
formation from the illustrative 
comparison. When results ave con- 
tradictory or inconsistent, =œ- will 
attempt to represent that fac« ard 
explore reasons for it. 


Question Wording 

Does the way in which a question 
is worded affect the ability of 
mentally retarded persons ty answer 
and the kinds of answers they iive? 
While we devoted more attenticn to 


` the effects of question format, 


we did compare a few alternative 
wordings of questions posed in the 
same format. (In the alternative 
formats section we will be discuss- 
ing several more instances in which, 
as part of a broader comparison, 
oppositely wordea questions with 
the same format were evaluated). 


USE OF EXAMPLES 

Consider first the sieataay of 
including in a question examples of 
the types of responses one is solic- 
iting. The use of examples (e.g., 
baseball or football as examples of 
the class of sports) might be ex- 


pected to aid mentally retarded 


persons in understanding a question 
by making the question more concrete. 
Alternatively, such examples might 
bias respondents, leading them to 
name the items used as examples 
more than they ordinarily would. 


a 
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Illustrative Comparison 
Two questions asked institu- 

tionalized adults to report if 
they played any indoor games, and 
if so, which games. One question, 
"Do you play any games indoors, 
like cards or checkers? (if yes) 
Which ones?" gave examples of 
indoor games. The other, "Do you 
play any games indoors? (if yes) 
Which ones?" solicited identical 
information but did not use ex- 
eed Analysis will be limited 

:> the effect of examples on re- 
sponses to the open-ended portion 
of each question. 


The use of examples increased 
responsiveness to the open-ended 
follow-up question. The percentage 
responding appropriately was 64.5% 
for the question with examples, as 
opposic to 52.0% for the question 
without examples. 


The wa: of examples also influenced 
the ge.2.:s mentioned in response to 
the open-ended question. Specifical- 
ly, only 22.2% of nine adults 
responding to both open-ended ques- 
tions mentioned checkers after the 
plain yes-no question, while 66.7% 
mentioned checkers after the ques- 
tion in which. checkers was included 
as an example, yielding a consis- 
tency figure for this category of 


were more likely to mention cards 
in response to the question in- 
cluding cards as an example (55.6%) 
than to the plain question (22.2%), 
producing a consistency figure of 
only 44.4%. Consistency—for_the 
remaining categories was generally 
high and was achieved primarily by 
not mentioning a game both times. 
In. short, respondents were far 


` more likely to mention a category 


if that category was cited as an 
example. 


—— Overall PEIRE E AI eshte 


and attendant across the seven 
catego:jes of response to the open- 


“Similarly,” ‘eesponaétite = 


ended question was 75.5% for the 
plain question and 72.4% for the 
question with examples. However, 


‘percentages of cases in which both 


mentioned a category averaged only 
10.2% and 4.8%, respectively, with 
most categories involving no such 
instances. For the critical 
categories used as examples, find- 
ings were inconsistent. For the 
checkers category, the use.of ex- 
amples apparently decreased agree- 
ment by inducing mentally retarded 
respondents to overclaim checkexs; 
agreement was 85.7% for the plain 
question and 60.0% for the question 
with examples. In the latter case, 
40.0% of client-attendant pairings 
involved the client mentioning 
checkers while the attendant did 
not. This trend was not apparent 
for the cards c tegory; agreement 
averaged 42.9% after the plain yes- 
no question and 53.3% after the 
question with examples, and incon- 
sistency was mixed rather than 
biased in one direction. 


Additional Comparisons © à 
- Two other pairs of questions 
asked of institutionalized adults 


examined the effects of examples. 


One pair asked about the client's 
participation :» “ports, using 


baseball and foxiball as examples 


in one phrasing, and another asked” 
about arts and crafts, mentioning 
ceramics and painting as examples. 
The effect of the use of examples 
on responsiveness to the open-ended 
follow-up is unclear. For the 


crafts question, examples apparently. 


increased responsiveness from 41.7% 
to 59.3%. Conversely, the sports 


question without examples generated 


greater responsiveness (75.0%) 
than the parallel question with 
examples (57.7%). 


These additional comparisons did 


____ support the. finding-that-citing 
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an example increases the frequency 


with whi:-}: that activity is men- 


tioned by mentally retarded persons. 
For the four examples used in these 
two pairs of questions, 56.1% of 
respondents, on the average, men- 
tioned the activity when it was 
included as an example, versus 44.3% 
who mentioned the activity when no 
examples were given. The pattern 
of inconsistenciés. generally re- 
flected this bias. toward overre- 
porting an activity used as example. 


Of these two pairs of questions, only 
the sports questions were asked of 
attendants. For the football cat- 
egory, agreement was 55.6% for 

the question with examples and 57.1% 
for the plain question. In both 
cases, most disagreements occurred 
because the client mentioned foot- 
ball and the atten. `t did not- 
Results for the baseball category 
were far more striking. Agreement 
was only 11.1% for the question 
with examples, and 77.8% of com- 
parisons were inconsistencies in 
which the client mentioned baseball 
and the attendant did not. For 

the plain question, agreement was 
far higher (71.4%). Thus, as for 
the indoor games questions, the 

use of examples decreased agree- 
ment for one of the activities 


‘used as an example pug not for 


the other. 


Conclusions 

The use of examples in the iaig 
ductory question had important ef- 
fects on clients' responses. Al- 
though the effect on responsiveness 


to the open-ended follow-up ques- 

tion is unclear, examples did in- 
crease the likelihood that respon- 
dents would mention the example 
activities. The existence of this 
biasing effect is clear from the ` 
consistency data, and client-attendant 
agreement figures suggest: 

that the use of examples sometimes 


_induces overclaiming_by mentally _ 





retarded persons, reducing response 
validity 7’ appears that the use 


pa 
Gi 
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of examples with open-ended ques- 
tions only added`to the problems 
inherent in the basic questioning 
technique. Examples appeared to 
arm interviewees with ready an- 
swers that they otherwise would 
not haye had. 


USE OF ALTERNATIVE TERMS 

We also examined the issue of 
question wording by testing the 
extent to which the language used 
to phrase a question affects rē- ` 
sponses. This strategy was ap- 
plied with four pairs of ques- 
tions about rules; one member of 
each pair asked "Are you allowed 
to...?" naming a specific be- 


‘havior, while the other member 


asked "Is it against the rules 
to...?" naming the same behavior. 


Illustrative Comparison . 

Two such questions, asked of 
institutionalized adults, were 
“Are you allowed to hit people?" 
and "Is it against the rules here 
to hit people?" There was no evi- 
dence that this difference in 
question wording altered respon- 
siveness, which was 70.7% for the 
"against" wording and 70.2% for 
the "allowed" wording. 


tions was 43.6%, a figure lower™ 
than that expected if responses 
to both questions were random. 
The great majority of respondents 
(87. 2%) arswered the "allowed" ques- 
tion "no," while responses were 
almost evenly split between "yes" 
and "no" in the case of the 
"against" wording. A very large 
fraction of clients, 46.2%, thus 
contradicted themselves by re- 
sponding "no" to both questions. 
This is in startling contrast to 
the more common result, where 
acquiescence on oppositely worded 


` 


and it suggests that acquiescence 
cannot always be expected on yes- 
no questions. The most probable 
explanation is that in the case 
of the. "allowed" questions, "no" 


‘was not only the correct response 


_ Overall consistency for these ques: 


(since hitting people is clearly 
prohibited) , but was also a\ highly 
ingrained socially desirable re- 
sponse. This response patten was 
carried over to the "against", ques- 
tions, probably because clients mis- 
understood the question and be 
lieved that "no" indicated that\ 
hitting people was forbidden. This 
interpretation, admittedly spec~ \ 
ulative, accounts for the differences 
in the marginals and the preponder— 
ance of "no-no" self contradictions. 


For the "against" wording, agree- 
ment with significant others was 
51.3%. Nearly.all agreement re- 
sulted because the client said 

"yes" (hitting people is against the 
rules) and the attendant concurred. 
Almost all disagreements (43.6% of 
comparisons) involved a "no" response 
by the client when the attendant 
indicated that hitting was forbidden. 
This again supports the idea that 
many clients misunderstood the 
question and attempted, through "no" 
responses, to give a socially 
desirable response. 





“was 76. 3%, with all agreement oc- 
.curring because client and attendant 


both said "no" (not allowed). 

Again, the major conclusion is that 
these questions; in direct contrast 
with the remainder of the yes-no 
questions included in our inter- 
views, did not give rise to acqui- 
escence, perhaps because clients 
had internalized strong negative 
evaluations of hitting. 


Additional Comparisons +: 
three additional pairs of ques- 
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questions causes a large proportion 
of "yes-yes" self contradictions, 


adults pitted "Is it against ...?" 


tions-poseđ- -of_institutionalized ~ 
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questions against "Are you ai- 
lowed...2?" questions. The activ- 
ities involved in these questions 
were ¢talling people ugly names, 


-leaving without asking, and stay- 


ing up late at night. Response 


` patterns to the first two ques- 


tions were virtually identical to 
those for the "hit people" ques- 
tions, but responses to the last 
pair of questions differed greatly 
from responses to the other three. 


Mean responsiveness for the three. 
additional "against" questions was 
68.4%, compared to 66.1% for the 
three "allowed" questions. This 
supports the conclusion that oppo- 
site wording did not alter respon- 
siveness. Consistency figures for 
the "ugly names" and "leave without 
asking" questions were 40.0% and 
57.2%, respectively, with incon- 
sistency predominantly due to re- 
sponding "no" to both forms of the 
question. Consistency for the 
"stay up late" question was 38.2%, 
with 23.5% of comparison inconsis- 


‘tencies due to saying "yes" both 


times, and 38.2% inconsistencies 

due to saying "no" both times. 
Clearly, staying up late is less 
strongly prohibited than the other 
behaviors, leading to far more 
responses by clients indicating that 
staying up late is allowed. It is 





_interesting that this factor reduced . 


question (an average of 32.5% of 
client-attendant pairings) involved 
a "no" response from the client and 
a "yes" response by the attendant. 


Conclusions \ 
Both consistency and agreement 

for these questions can be easily 

interpreted as arising from a 


strong tendency to say "no" on the . 


part of clients. Why does the usual 
yea~saying not appear? The simplest 
explanation is that there is a strong 
motive to give socially desirable 
answers; that is, it is highly de- 
sirable to express the understanding 
that prohibited behaviors are pro- 
hibited. This accounts for the high 
proportion of "no" responses to 

the "allowed" questions. Respondents 
appeared to understand the rules well 
enough to give a negative response 
to negative behavior. Whether they 
understood the "allowed" phrasing 

is less clear. One must assume that 
many clients misunderstood the 
"against" questions, believing that 
a "no" response was again the way to 
express disapproval 7of undesirable 
behavior. - This interpretation at 
least raises the possibility that 
much of the acquiescence observed 
throughout our interviews may be 
partially accounted for by social 
desirability, for questions were 





consistency. 


For the three additional "against" 
questions, average agreement with 
attendants was 54.1%. Agreement 
was far higher, averaging 80.8%, 

for the "allowed" wording. In 

the latter case, clients' "no" re- 
sponses coincided with the preferred 
response of attendants. For the 
"against" questions, clients’ at- 
tempts to indicate prohibition 
through "ic" responses operated in 
opposition to attendants' responses, 
reducing agreement. This is sup- 


~=bypical-ly--worded_so_that.."yes"..was....... 





the socially desirable response. 
It may be that a tendency toward 
agreement with socially accepted 
views (see discussion of Rosen et 


‘al., 1977) is.in fact the general 


response set in mentally retarded 
persons, and that acquiescence “` 
to most yes-no questions and nay- 
saying to the rules questions are. 
simply specific manifestations of 
that broader tendency.’ Alterna- 
tively, respondents may have been 
so well-versed on the prohibitions 
asked about in these questions 





““portea by the observation that most 


disagreements for the "against" 





acquiescence were overcome. 


that their usual_tendencies toward 


Regardless: of the factors underiy- 


ing clients' responses, one major 
conclusion emerges clearly from 
these comparisons: 


whichis derived from interviews 
differs as a function of question 
wording. For the four "against" 
questions, an average of 61.2% of 


clients indicated that the behaviors 


are prohibited. For the four 
"allowed" questions, an average of . 


75.4% of clients indicated that | the 


behaviors are prohibited.. In 


neither case do clients agree fully 
with attendants, about 85% of whom 
indicated that the stated behaviors 
In short, because. 


are prohibited. 
the responses of mentally retarded 
individuals to yes-no questions 


are not entirely valid, the picture 


of the sample derived from inter- 
view data depends largely on how 
questions happen to be worded. 


SLIGHT CHANGES IN QUESTION WORDING 


Serendipity gave us some insight 
into the effects of question wording 
that we would not otherwise haye had. 


closely related questions, " 
decides what chores you do?) 


was startling. 
Responsiveness ‘to these 
ended questions was ide tical at 
34.5%. Regarding consistency, the 
fact that the first of the two 
questions was repeateA fv the two 
forms of the intervi¢w gives us a 
baseline estimate off response 
reliability over a period of a 
week was 83.3%. Fof the two ` 
comparisons betweén the different 
wordings of the question, consis- 
tency figures wére much lower ~- 
62.6% and 73.3%. While the 
__——-number-of—per ons—responding—was- 








the picture of 
a group of mentally retarded persons - 
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ures with the 83.3% figure for test- 
retest reliability suggests that 
‘very small changes in question word- - 
ing’may have effects on clients’ re- 
sponses. 


Agreement figures for the two "you 

do" ‘questions were 88.2% and 75.0%. 

Agreement for the "to do" wording 

was’ considerably lower (70.6%). 

we amb the "to do" question was 
e ambiguous. 


i 


onclusions ; 

Clearly, even very slight changes 
in question wording can have impor- 
tant effects on the reliability and 
validity of the responses of mentally 


. retarded respondents. 
| 


USE OF QUANTITATIVE QUALIFERS 
Included in follow-up interviews 
‘ with the severely retarded sample 
were five yes-no questions and one 
either-or question with picture al~ 
ternatives, all dealing with whether 
or not the client makes his or her bed. 


The questions, which used quantita- 
tive modifiers to introduce slight 
differences in meaning, were as 
follows: 
Do you always make. your bed here 
in the cottage? 
Most days, do you make your bed 
here in the cottage? . 
__Do you; usually..make.your bed _- 
here in /the etase: 
Do you ‘sometimes make your bed 
here in the cottage? 
Do. you ever make your bed here 
in’ the cottage? 
This picture shows a boy/girl 
who makes his/her bed here in 
‘the cottage. This picture shows 
‘a boy/girl who does not ever 
make his/her bed here. Which 
boy/girl is most like you? 
Point to the picture. 


Responsiveness for the five yes-no 
questions ranged from 79.3% to 
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mparison of these fig- 
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Table 8.1 % "Yes" Responses to Bed-making Questions 
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Always 











Most days 

usually 

Sometimes E : as 
Ever (Yes-no) 

Ever (Picture either-or) r ' 





89.7%, averaging 84.4%. There is 
no indication that responsiveness 
differed systematically as a func- 


Conclusions 
Perhaps the conclusion best 
justified by the data is that clients 


tion of the quantitative modifier 
used. Responsiveness for the - 
either-or question using pictures 
was 89.7%. 


were relatively insensitive to the 
quantitative shadings involved in 
these ‘questions - they tended to re- 
spond in the affirmative reguardless 


of the quantitative modifier used. 
This suggests that laborious efforts 
to refine questions through quanti- 
tative modifiers may not have the 
effects that question designers in- 
tended them to have. 


Regarding consistency, the best 
strategy for extracting meaningful 
information from these comparisons 
is a kind of hierarchical analysis; 
ıf a client responds "yes" to the 
"always" question, then he or she 
must necessarily respond "yes" to 
the other questions to remain 
consistent. Table 8.1 shows the 
percentage of "yes" responses to 
the six bed-making questions. As 
can be seen, no’ more respo-7cuts 
answered "yes" to the "usually" 

or "sometimes" question than to the 
"always" question. .This calls 
‘into question the. validity of re- 
sponses. Although it is logically 
‘possible that clients virtually 
all make their beds every day, re- 
sponses to/the pictorial | either-or 
question. suggest otherwise. Be- 
cause res onses/ to these questions 
represented subtle quantitative 
differences, agreement with sig- 


SPECIFIC VERSUS VAGUE WORDING 

Two comparisons of open-ended 
follow-up questions in the institution- 
alized adult sample compared general 
or vague question wording witha more ~: 
specific wording which restated the 
main, content of the question. These 
comparisons suggested that type of word- 


4 


ing had little effect on consistency or. / 


agreement, but had a clear effect on re- 
sponsiveness; the specific wording maxX- . 
imized responsiveness. For example, ās 

a follow-up to a question about attitudes 
toward, living in the institution, clients 
were asked either "Why do you feel- 


that way?" or Why do you feel that way 
about living here?" responsiveness to 
nificant others is not a useful the follow-up using the, former word- 

index of validity. ing was 26.2%, as compared to 54. 18 for 


race ey a 
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content of the question. 


the latter (more Specific) wording. 
Results were identical for a follow-up 
question dealing with clients’ 
attitudes towards the food, with the 
responsiveness figure favoring the 


' more specifically worded question, 43.9% 


to 30.6%. In short, it is clear 


that! if maximizing responsiveness 
to-open-ended follow-ups is the 
desired goal, the interviewer should 
use [wordings which repeat the main 
This 
strategy does not appear to lessen 
validity in any way, and it has the 
decided advantage of reminding 

ic aaa of the question content. 


THE USE OF PROBES WITH OPEN-ENDED 


‘QUESTIONS 


‘As the data presented thus far 
suggest, the researcher or program 


*planner:who wishes to interview 


Ps 
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-- -——to-increase-the-amount—of—informa~ 


mentally retarded individuals faces 
a dilemma: highly structured ques- 
tion formats such as yes-no ques- 
tions maximize responsiveness but 
tend to introduce response biases 
which seriously reduce response 
validity; less structured formats 
such as open-ended questions, on 
the other hand, can be answered 

by few and elicit little informa- 
‘tion from them. In crder to find 
ways to'’improve open-ended questions, 
we at times probed "what else" as 
respondents answered *hem, hoping 


Illustrative Comparison 

Institutionalized adults were 
asked the questions "What do you 
usually do for fun when you are by 
yourself?" and "What do you usually 
do for fun when you are with someone?" 
(probe: "What else?" until client 
can think of no more). 


Consistency of responding to these’ 
questions was evaluated in terms of 


_both the number of responses given 


to each question and the content of 
the particular responses given. In 
terms of numbers of responses, clients 


averaged 5.0 responses to the ques- 
tions with probe and 2.9 responses 
to the question without probe. 
Clearly, the use of the probe in- 
creases the number of responses 
given by the clients who could 
answer the questions. 


Consistency of response content, 
averaged over the ten categories 
into which responses were coded, 
was 80.0%. However, most of this 
observed consistency occurred 
because -clients did not mention a 
category in response to either for- 
mat; only 8.6% of comparisons were 
consistent in that the same category 
was mentioned both-.times. Of more 
interest is the fact. that most 
inconsistencies occurred when the 
client mentioned a category in 
response to the question with 
_probe and failed to mention the: 





tion gained. The following section 
is a comparison of responses to 
open-ended questions using such 
probes and to otherwise identical 
questions with no probing. Because 
the probe follows the first re- 
sponse, the formats are identical 
with respect to the percentage of 
clients responding appropriately 

to the question, and analysis of 
responsiveness is omitted from the 
following discussion. ; 


same category when no probe was 
given. Of all comparisons, 15.0% 
(or three-fourths of all incon- 
sistencies) were inconsistencies 
of this type. 


Client-attendant agreement can also 
be evaluated either by the number 
of responses given, or by the re- 
sponse categories used. For the 
question without probe, clients 
averaged 2. 5 responses versus 2.7 
for attendants. For the question 
with probe, clients averaged 4.7 
responses versus 2.6 for attendants. 
Thus, the use of the probe induced 
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clients to mention far more acti- 
vities than did their attendants; 
this finding at least suggests 

that the probe technique may, by | 
placing demands on the client, cause 
overreporting. 


Agreement between attendants' and 
clients' response to the question 
without probe, averaged across the 
ten response categories, was 82. 3%. 
For the question with probe, 

average agreement across the ten 
response categories was 74.6%. Thus, 
the use of the probe apparently re- 
duced agreement, most disagreement 
occurring because the client men- 
tioned a category which was not 
mentioned by the attendant. All 
client-attendant disagreements in 
the probe format (17.7% of all paired 
responses) were inconsistencies of 

In short, the use of 

the probe reduced agreement by in- 
ducing clients to mention categories 
not mentioned by attendants. 


Additional Comparisons 
Institutionalized adults were 
asked three additional pairs of open- 
ended questions comparing the prob- 
ing approach with identical ques- . 
tions without probing. These ques- 
tions asked, "What do you and your 

friends usually do together?" 
"Where do you usually get your 
money?" and "What are they teaching 


Conclusions 

A remarkably clear pattern 
emerged from these comparisons. 
First, the use of the probe tech- 
nique was effective in eliciting 
more responses from mentally re- 
tarded respondents. Unfortunately, 
this had the effect of reducing 
agreement. It appears that the 
probe created a demand which was 
met by clients,- regardless of 
whether the information provided 
was valid or not. The net effect 
is that probing seems to cause 
inappropriate oyerresponding, re~- 
ducing response validity. There - 
was also some evidence that under- 
reporting. occurs when no probe is 
used. Possibly, the optimal ap- 
proach would be to probe following 
one or two, responses, rather than 
continuing to probe indefinitely. 


The disastrous effects of probing 
ad infinitum were clear in several 
of the interviews. One Severely 
retarded male, asked about: what he 
and his friends usually do to- 
gether, apparently felt compelled 
to repeat himself in response to 
each, "What else?" His responses 
were "play round," "play around," 
"go outside and play," and "go 
play," in that order. Clearly 

no new information was obtained; 
the only-effect of probing was to 
force him to change his wording 
a_little_in hopes of convincing 
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you there?” a question which was 
asked only of respondents who had 
answered "yes" when asked "Do you 
go to school or take classes?" 
Results from additional comparisons 
of the probe and no probe formats 
were remarkably similar. In*‘each 
and every case, mentally retarded 
individuals gave more responses 
when the interviewer probed "What 
else?" after each response and*in ~ 
each case the use of the probe de- 
creased agreement. 











the interviewer that he had new 
news. Another, in response to 
the question, "Where do you get 
your money?"` said, "I work." 
However, «he then interpreted the , 
"where else?" probes that followed 
as asking where else he worked, 
and then responded, in turn, "at 
the kichen” "cottages," and "in 
here." 
the probes as a license to play 
word games, ran cff lists of over 
10 activities, clearly doing so` 
to meet the interviewer's demands _ 
vith cach new probe. 
‘ne repeated himself somewhat, ~ 
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Still others, interpreting” 


One, though 
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came up with the staggering total 

of 27 responses to 27 "What else's." 
His last response had to do with - 
cleaning, and from then on he took 
the "What else's" to refer to other 
things he cleaned, giving the inter- 
viewer a much-welcomed opportunity 
to go on to the next question. 


Multiple-Choice Questions 


THE MEANING OF QUANTITATIVE RESPONSE 
OPTIONS í 

We used many multiple choice 
questions offering response alter- 
natives like "a lot" and “some," 
which differ along a quantitative 
dimension, in order to collect in- 
formation about extents of involve- 
ment in various activities. The 
many problems we encountered in the 
use of quantitative multiple choice 
questions in interviewing mentally 
retarded persons may be a result 
of the difficulty those individuals 
have in attributing consistent mean- 
ing to quantitative words and 
phrases. One, comparison will serve 
to illustrate this phenomenon. 
We asked institutionalized adults 
"How many friends do you have? A 
lot, some, not many, none?" and 
simply "How many friends do you 
have?" Respondents were more re- 
sponsive to the multiple choice 
question (55.2%) than to the open- 
ended question (39.7%). 


Regarding consistency, the numbers 
offered in response to the open- 
ended item did not correspond well 
to the categories endorsed in 
responding to the multiple choice 
question. For example, the 64.7% 

of respondents who answered.from one 
to six friends on the open-ended 
question answered anywhere from 
"none" to "a lot" when asked the 
multiple choice form. Among respon- 
dents who said they had seven or 
more friends in response to the 
open-ended question, responses to 


1 


the multiple choice form were divided 
between "a lot" and "not many." 


In view of the fact that these quan- 
titative terms appeared to have very 
diverse meanings among institution- ` 
alized adults, it is not surprising 
that clients and attendants were not 
in consistently close agreement. For 
example, in response to the multiple 
choice question, 18.8% of residents 
said they had no friends while their 
attendant said they had five or 

more friends; another 21.9% said 
they had "a lot" of friends while 
the attendant was reporting four 

or fewer friends. For the open- 
ended question, 17.4% of comparisons 
involved clients saying they had 

one. to three friends, while the 
attendant reported over ten friends. 
Another 17.4% of comparisons in- 
volved clients saying they had no 
friends while the attendant said 
that they had at least one friend. 


Conclusions p g 

In short, the evidence suyqests 
that the use of quantitative con- 
cepts by our respondents was very 
inconsistent, that the meanings 
of these terms apparently differed 
greatly from person to person. 
In view of this fact, the disappoint- 
ing results achieved using quanti- 
tative multiple choice questions 
are not surprising. Sample 8.2 il- 
lustrates zome of the difficulties 
our interviewees had with quanti- 
tative concepts. _ 


ALTERNATIVE „MULTIPLE CHOICE FORMATS 
Quantitative multiple choice 
questions were included in our 
interviews largely in an attempt 
to get an idea of the extent of 
involvement in various activities 
on the part of our respondents. 
However, we were from the beginning 
concerned that four response alter- 
natives might tend to confuse 
mentally retarded individuals. In 
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Sample 8.2 


A Sampling of Problems with Quantitative Concepts 


l. Q: 
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Charlotte 


How often do you leave here to go see your family: A lot, 
\ 

sometimes, not much, or never? 

t 

{Respondent said yes as each alternative was given.) 

i \ 


How many people sleep in your bedroom? | 


How many people sleep in your bedroom? 
We got a whole bunch of beds tack there. I can't count 
‘em all. Not righ: now . i 
Are you usually happy or sad? E 
I'm always happy sometimes. / 
Are you usually sad? 

Yes, when somebody hits me I am. | of the 


question, he said, “No, not very much, ") / 
/ 
How much ao you watch TV - a lot, some, not’ much or never? 


Some, much. 


` (repeated) | 


Never. 
Do you watch TV a lot, scime or not much? 
Some, not a lot, much. 


(repeated) 


I'm going to watch Elvis Presley tonight. 
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8. Q: How much do you read books, magazines, or newspapers - 
Never, not much, some, or a lot? 

A: I can't read. I'm sorry, but I can't. 
Q: (Repeated since none of the four options was chosen) 
A: A lots: 

And finally, a reconsideration of knee-jerk acquiescence: 

9. Q: Could you buy à new TV with $10.00? 

E A: Yeah, I do. No, I can't. You can't buy a TV with that 


money. You pulling my leg off. 
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Table 8.2: Yes-No Plus Three-Choice Multiple Choice ys. Four-Choice Multiple Choice 





ce a 
i li 
C oE a 

















f 

Í 
i 

i 


Four-Choicé Yes-No & Three Choice 





Multiple Choice Multiple Choice 
j , 
Content 3 RESP /% AGREE % RESP % AGREE 
iA 

How often do you leave here to go see your 78.0%“ 50.0% 66.7% 52.48 

family? i 

How often does anybody in your family come 15.63 30.0% 

to see you? j 
ae tr a I 

| 

How much do you watch TV? . 55,28 22.2% 
ee 7 : ae ae aa 

How often do you go to church? i -48.33 N i a 
E a 

How often do you go to stores? 49.1% NA 

Bisa Rea il Se eae a eee 

How much do you listen to the radio or, record | 41.3% 45.8% 

player? 
a a eee ee RE renee mer 

How often do you go out to th. movies? 49.1% 5.9% 
N A E e ee aes ieee 

How much do you read books, magazines or = | 52.6% 18.1% 

newspapers? 


How often do you gu out tc eat? 155,24 16.1% 
i Average: | 56.08 C338 | 57.68 > 37.58 
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an effort to simplify the task 
demands but still obtain quantita- 
tive information, we combined some 
yes-no questions with three-choice 
multiple choice questions asked 

of only those respondents ,who 
answered the yes-no question "yes." 
For example, we asked "Do you leave 
here to go see your family? (if 
yes) Do you go 3ee them a lot, 
sometimes, or not much? The 
comparisons that follow -ompare 

the four-choice’ multiple choice 
format with the yes-no followed by 
contingent three-choice multiple 
choice format. In the analysis 
that follows, responsiveness figures 
have been transformed so that re- 
sponsiveness for the two formats 
can be directly compared, and con- 
sistency and agreement figures have 
been similarly adjusted (with a 
"no" response to the yes-no question 


. coded as a "never" response to the 


fourrchoice multiple choice ques- 
tion). 


Illustrative Comparison 

_ One comparison of these two 
formats consisted of asking insti- 
tutionalized adults both "How much 
do you read bc igazines, or 
newspapers? Ne -, not much, some, 
or a lot?" and "Do you read books, 
Magazines, or newspapers? (if yes) 
Do you read books, magazines, or 
newspapers a lot, some or not much?" 


A total of 48.3% of respondents 
responded appropriately to the yes- 
no question and subsequent three- 
choice multiple choice question. 
This is slightly less than the 
52.6% who gave appropriate answers 
to the four-choice question. 


Of 26 subjects who respgiitied ap- 


_propriately to both formats, only 


50.0% gave consistent answers. The 
patteri of ince ‘stency is of 
interest: in 01.35€ to both 
questions, the ~-spiuse alternative 
most frequently chosen was the last- 


‘mentioned. Thus, 11.5% of compar- | 


l 


isons were inconsistencies due to 
the client choosing the last option i 
in both cases. This suggests that _ 
consistency was reduced in part by 

a last option response bias. In 
addition, 80.8% of clients re- 
sponded "yes" to the yes-no ques- 
tion, indicating that they at least 
sometimes read. In response to the 
four-choice multiple’ choice ques- 
tion, only 69.3% of clients in- 
dicated that they at least some- 
times read. This contrast reflects 
acquiescence in response to the 
yes-no format. 


The four-choice multiple choice 
question produced only 18.1% agree- 
ment with attendants. Most dis- 
agreements (50.5% of comparisons) 
occurred because the client re- 
sponded "a lot" (the last option) 
and the attendant said something 
less. Agreement for the yes-no plus 
three-choice multiple choice ques- 
tion was. slightly higher at 32.0%. 
Of all comparisons with attendants, 


' 24.0% were “not much" responses 


from the client ( the last option), 
coupled with a response of either 
"never" or "some" by the attendant. 
In short, 4. nent is low for both 
formats, with ti. Pattern of in- 
consistencies indicative of some 
last option response bias on the 
part of the residents. i 


Additional Compariso 

A total of nine EREE ATE 
of four-choice multiple choice ques- 
tions with yes-no plus three-choice 
multiple choice questions were con- 
ducted. Table 8.2 depicts respon- 
siveness, consistency, and agreement 
data for--all- nine comparisong. Al- 
though the specific comparison dis- 
cussed above is generally repre- 
sentative these sets of ques- 
tions, it -~ ‘%2>r that there is a 
lot of variability within the set. 
Nevertheless, the general conclu- 
sions suggested by the "read" ques- 
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tions are unaltered; specifically, 
neither format appears attyactive. 
Responsiveness, consisteucy, and 
agreement are poor for both formats, 
neither appears to offer a definite 
advantage over the other. Agreement in 
particular was poor for both formats 
37.5% for the yes-no plus follow-up 
approach versus 32.3% for the four- 
choice questions. Only the questions 
about church attendance appeared to 
work well, possibly because there 

is a relatively clear external 
standard suggesting that once a 

week or more is "a lot." 


It should be pointed out that the 
last option response bias discussed 
above’ was observed frequently but 
not inevitably within this set. 

It appears that mentally retarded 
respondents are often biased in 
favor of the last-mentioned al- 
ternative ,/but that this set is 
weak ont that it is frequently 
overshadowed ‘by other factors. The 
important point is that these 
quantitative multiple choice ques- 
tions do not a useful informa- 
tion. 


VERBAL MULTIPLE CHOICE QUESTIONS vs. 
MULTIPLE CHOICE QUESTIONS 
WITH PICTURES 

A direct comparison of standard 
multiple choice questions with 
multiple choice questions offering 
pictures as response alternatives 
was undertaken to see if one format 
had a definite advantage. The use 
of pictures as response alternatives 
was primarily intended to elicit 
information from clients with few 
verbal skills. These. comparisons 
involved subjective questions. 


Illustrative Comparison 
Institutionalized adults were 
asked,"How do you like living here: 
a lot, some, not much, or not at 
all?" and "Which picture shows how 


you feel about living here? Point 
to the picture." (Sad face followed 
by somewhat sad, somewhat happy, 
and happy faces.) i 


Consistent with prior observations 
regarding responsiveness to picture 
formats, the use of pictures in- 
creased responsiveness. In response 
to the picture alternatives, 70.7% 
of clients responded appropriately 
by pointing; for the verbal multiple 
choice question, responsiveness was 
50.9%. 


Responding across the two formats 
of the multiple choice question 
was very inconsistent;’the overall 
consistency figure was only 21.4%. 
As to the pattern of this incon- 
sistency, it appears that the use 
of pictures tended to spread re- 
sponses across response alternatives. 
Responding to the verbal format 
was biased toward the "a lot" end 
of the continuum, with 35.7% of 
clients replying "a lot" (the first 
option). Responding to the pic- 
ture format did not favor one end 
of the response continuum, as ` 
32.1% of residents answered "a lot" 
and 39.3% replied "not at all." 
This could reflect a first option 
response bias in the case of 
verbal alternatives, or Simply a 
greater tendency toward random re- 
sponding in the case of picture 
alternatives. Since a bias toward 
choosing the last option on mul-_ 
tiple choice questions has been 
observed elsewhere, we favor the 
latter interpretation. 


Although these questions were sub- 
jective, we asked attendants to 
characterize residents' satisfaction 
with the institution. Agreement for 
the picture format was 21.3%. 
Agreement for .the word format was 
slightly higher, at 32.3%. In both 
cases, inconsistencies arose pri- 
marily because attendants rated | 
clients as having more positive } 
feelings than were reported by ° 
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the clients themselves. This may 
well indicate that there is a real 
difference of opinion between 
clients and attendants on the issue 
of satisfaction with the institu- 
tion. However, con. -ibis “cy data 
still suggest that neither of these 
formats successfully elicited valid 
answers, and that the picture format, 
while it enabled many persons to 
respond, is particularly suspect. 


Additional Comparisons 

Two additional comparisons of 
picture and verbal multiple. choice 
questions were conducted, but with- 
out data on agreement with attend- 
ants. These questions involved 
satisfaction with the food and 
with the people at the institution. 
In both additional comparisons, 
the use of pictures was associated 
with higher responsiveness than was 
the verbal questions. These com- 
parisons were 67.9% vs. 50.0%, 
and 67.9% vs. 55.2%, respectively. 
Consistency figures for the questions 
regarding food and people were 33.3% 
and 37.8%, respectively. This very 
low consistency, indicative of a 
lack of reliability of responses 
to these quantitative multiple 
choice questions, suggests that 


‘ neither format successfully gen- 
-erated reliable information. 


Conciusions 

Clearly, the use of pictures as 
response alternatives increased re- 
sponsiveness to these multiple choice 
questions. However, consistency 
was so low as to suggest that neither 
format is very useful. Based on 
agreement. data and the pattern of 
inzonsistencies obsérved, it appears 
that the gain“in responsiveness 


achieved by using pictures is not —__ 


accompanied by an increase in re- 
liability and validity. ‘Sample 8.3 
illustrates some of the problems 
with these picture choice questions. 


These discouraging data are not 
surprising in view of the fact 
(discussed in’Chapter 6) that test- 
retest reliability for one arovn 
of picture multiple choice 

tions averaged only 46.2%. 

can be compared roughly with . .-: 
reliability for verbal multiple 
choice questions on different 
topics, as reported in Chapter 6.) 
Responses of mentally retarded 
persons to quantitative multiple 
choice questions, whether verbal 


or pictorial, appear to be in- 


herently unreliable. 


DISCRETE MULTIPLE CHOICE QUESTIONS 
The multiple choice questions 
previously discussed, which appear 
unsatisfactory for use with men-. 
tally retarded populations, in- ~ 
volved subjective judgments on 
the part of respondents and used 
response alternatives which differed 
along a quantitative dimension. 
Another group of multiple choice 
questions in our interviews dealt 
with factual information and offered 
response alternatives which differed 
in a qualitative manner--that is, 
discrete response optior™. =- 
Discrete multiple choice questions 
in both “point to the picture" and 
verbal response forn. 
pared directly with open-enucd 
questions soliciting the same 
information. 


Illustrative Comparison 

Community children were asked 
how they traveled to school using, 
each of the following three ap- 
proaches: “Here are some ways 
people get to school. (Pointing. to 
pictures). They take the bus, 
somebody drives t ^+ school in 
a car, they wala, or v Tour to 
school on their bicycles. Which 
way do you get to school most days? 
Point to the picture."; “Most 
days, how do you get to school?"; 
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Sample 8.3 


me 


od Reool+s with Picture Choice Questions \ 


Failure Story: 


Q: 


Which picture shows how you feel about living here? 
` Point to the picture. 

(pointed to "not at all" saddest face of four faces) 
Why do you feel that way? | 

I don't like to feel that way. 

(repeated) | 


I don't feel that way. 


Weird Story 


Q: 


O. 
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Which picture shows how you feel’ about the people here? Point 
_to the picture. 
~~ 

(did not point) They're not good. They. . . (stopped) 

(repeated) 

(did not point) sues they get mad about the way she acts - she'll 
do anything I say (reference not clear) 

Why do you feel that way? | 

They don't assume _I harp on me to say. They don't understand 


what I say. 
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Success Story: i ` 
Q: which picture shows how you feel about the people here. 
Point: to the picture. 
A (pointed to “not at all" saddest face) 
Q: Why wo you feel that way? 
_A: He looks like he's crying, tears falling from his vye». 
People wane makii fun of him. He wants to go home to his parents. 
Note: Unfortunately this success story is the exception. 
Typically respondents had difficulty giving consistent responses 


to these picture choice questions about satisfaction. 
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and "Most days, do you get to 
school on the bus, in a’car, by 
walking, or riding a bicycle?" 


Responsiveness to the picture mul- 
tiple choice question was 100%; all 
children asked the question were 
able to ::spond appropriately by 
pointing to one of the pictures. 
For the verbal multiple choice 
question, responsiveness was 34.5%. 


..The-open-ended. question elicited 


the lowest percentage oi responses 
From children (73.6%). Clearly, 
the use of multiple choice que” ons 


_in general and of picture mul 


choice in particular optimize: 
sponsiveness. 


Consistency for comparisons using 
the discrete multiple choice format 
was exceptionally high. Comparison 
of the picture multiple choice 
format with the open-ended question 
produced a consistency figure of 
95.2% for 41 response pairs. In 
the comparisons of the verbal mul- 
tiple choice and open-ended ques- 
tions, 95.1% of children responded 
consistently. The comp2rison be- 
tween the. two multiple choice ques- 
tions produced a consistency figure 
of 86.9%. For the latter two 
comparisons, most inconsistencies 
‘arose because children often 

chose the last option in the verbal 
multiple choice format, again sug- 
gesting a last option bias. How- 
ever, this interpretation must be 
tempered by the realization that 
consistency was so high that pat- 
terns of inceusistency involved 
very few subjects; the major 
conclusion remains that consistency 
was remarkably ae for these ques- 
tions. 


Parent-chiid: agreement figures for 
the picture multiple choice (85.5%) 
and the open-ended question (85.4%) 
were almost the same. There was 
slightly less agreement between 
parents and children on the verbal © 
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multiple choice format (77.4%). The: 
only pattern observed in the disa- i 
greements was a slight last option 
bias on the verbal multiple choice 
question; 13.2% of pairs disagreed 
because the child said "bike" (the 
last option) while the parents said 
either "car". or "hus." .In this 
case, then, the picture multiple 
choice question is preferable on 
grounds of POLN Eee ponsiveness and 
validity. are. ares 


Addi? ¿onal Comparisons 

There was another comparison, 
also involving community children, 
of four question: asking in what 
kind of place the child lived 
(house, apartment building, trailer 
house, or duplex). The questions 
were discrete multiple choice, pic- 
ture and verbal, with response al- 
ternatives presented in original 
and alternative orders as a test 
for order effects. 


Average responsiveness for the pic- 
ture choice questions.was 96.5%, 

as opposed to 81.8% for the verbal 
questions. Consistency of response 
between the twc verbal choice formats 
was slightly higher than that for 


> the picture questions--85.7% as 


compared to 78.2%. For the two 
comparisons between word and picture / 
formats presenting options in the : 
same order, consistency averaged 
81.8%. The two comparisons be- 
tween verbal and picture formats 
which presented options in different 
orders yielded a similar average con- 
sistency figure of 78.8%. These 
consistency data suggest that (1) 

the superiority of the picture ques- 
tions observed earlier was not sup- 
ported, and (2) order biases’ are not 
particularly important in these for- 
mats, Since different order compari- 
sons are about as consistent as same 
order comparisons. Average agreement 
between parents and their children 
for the verbal choice questions was 


/ 


ii ` 
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81.8%, while average agreement for 
the picture format questions was 
70.5%. This again reverses the 
conclusion drawn from the transpor- 


tation questions, but it is not clear 


why the validity of responses to the 
picture choice question was lower 
here. 


Conclusions 

One conclusion which can be drawn 
from these comparisons’ is that dis- 
crete multiple choice questions about 
basic facts differ strikingly. from 
their quantitative counterparts. 
These questions yielded high respon- 
siveness, high consistency, and 


high agreement relative to most other 


formats. Comparisons between pic- 
ture choice and verbal choice ap- ` 
proaches resulted in inconsistent 
findings, so that a global judgment 
of their relative merits is impos- 
sible. There is some evidence for 

a last option bias in verbal mul- 
tiple choice questions, but this 
phenomenon is unstable. To the 
extent that order bias affects 
verbal choice questions, it at 

least does not consistently reduce 
their validity below that of picture 
choice questions. However, since 
picture multiple choice questions 
consistently maximize responsiveness 
and there is no reason to believe 
that they are consistently less 
valid than verbai choice questions, 
picture multiple choice would appear 
to be an excellent method for. elic- 
iting factual information from. 
mentally retarded respondents. Un- 
fortunately, we did not test the 
discrete multiple choice format with 
other content areas, so it is im- 
possible to determine whether the 
success found here would generalize 
to other topic areas, or whether re- 
tarded childrer simply have solid 
enough understanding of how they get 


‘to school and where they live that 
almost any format would work. 


e 


` Yes-No Vs. Open-Ended Questions 


LISTING QUESTIONS 

One type of information which 
we sougire ‘to obtain from mentally 
retarded persons was enumerations 
of. activities, problems, interests, 
and so on. The decision faced by 


- an interviewer interested in ob- 


taining such information hinges 

on the issue of question structure. 
Should one choose highly structured 
formats and run the risk of re- 
sponse biases Such as acquiescence 
or should one choose open-ended 
formats and run the risk that few 
interviewees Will be able to an- 
swer the questions? To evaluate 
this problem, we directly compared 
open-ended questions asking for 
active enumeration of responses 
with sets of yes-no questions in- 
tended to obtain the same informa- 
tion. i 


Pa 


One comparison of these two 
formats, drawn’ from the institu- / 
tionalized children sample, ac- / 
curately represents finding for e 
group of comparisons as a whole’. 
Institutionalized children were 
asked eight yes-no questions/about 
sports in which they might par- 
ticipate (for example, "Do you 
play football?"). Residents were 
also asked an open-ended question 
soliciting the same information: 
"Do ycu play any sports?" (if yes) 
Which ones?" Respo.ses te the 
open-ended format were subsequently 
coded into categories correspond- 
ing. to the yes-no questions, SO 
that each client responded "yes" 
or "no" when asked if he or she 
plays a given sport and was cate- 
gorized as -having Mentioned or not 
mentioned that sport in response to 
thé open-ended question. Attendants 
were asked a parallel list of yes- 
no questions ("Does this client play 
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A 
any of these sports: 


} Football? 3 
Baseball? and sý on). 


Average responsiveness to the eight . 
yes-no questiors was 85.6%. This 
compares favorably with the 55.2% 
figure for the open-ended question. 
Consistency Jas evaluated by aver- 
aging consistency across the eight 
pairings of/a yes-no question with 
its corresponding response category. 
Average consistency was 49.7%, with 
inconsistency resulting almost ex- 
clusively from pairings in which 

the client replied "yes" on the yes- 
no format but failed to mention that 
response category in response to 

the open-ended question. Of all 
comparisons, fully 48.1% were incon- 
sistencies of this type. As well 

as ‘Alustrating the ubiquitous im- 
pact of acquiescencé, this finding 
highlights the possibility that 
underreporting is a serious problem 
in the’ response of mentally retarded 
persons to open-ended questions. 
‘Examination ot data regarding agree- 
ment with attendants is necessary to 
determine which effect is stronger. 


Client-attendant agreement was cal- 
culated for the eight-yes-no ques- 
tions and for the eight categories 
of response to the open-ended ques- 
tions. Averzge agreement for the 
yes-no questions was 52.1%, as Op- 
posed to 60.1% for the open- -ended 
question. For the yes~-no questions, 
42.3% of all pairings were dis~ 
agreements in which the child said 


"yes" while the attendant said 
"no," and an additional 5.6% were 
disagreements in which the child 
said "no" and the attendant 

"yes." Again, the overwhelming im- 
pact of acquiescence is evident. 
For the open-ended question, 31.9% 
of client-attendant pairs disagreed 
jn that the attendant said "yes" 
regarding a sport not mentioned by 
the retarded respondent. Only 8.1% 


‘of pairings were disagreen-.its in 


--to underreport their activities 
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which the child mentioned a sport 
not indicated by the attendant. 
Thus, retarded interviewees tende 


in response to the open-ended ques- N 
tion. In short, acquiescence leads 
to overreporting in response to 
yes-no lists and clients are prone 
to underreport in response » open- 
ended questions; the latter bias 
may be slightly less dramatic. 


- 


Additional Comparisons 

Four additional sets of ques- 
tions, asked ,of institutionalized 
adults and imstitutionalized chil- 
dren, compafed open-ended and yes- 
no listin questions. These sets 
of questiéns solicited information 
client's problems, actions 
of othets that bother the client, 
nice tHings people do for the ao 
and participation in arts and 
crafts. 







In each comparison; far more clients 
gave appropriate responses to the 
yes-no questions than the open-ended 
questions. The responsiveness 
figures for the four yes~no ques- 
tions ranged approximately from 
60-80%, while those for the open- 
ended questions ranged from 20-508. 
The superiority of yes-no questions 
in eliciting avpropriate responses 
is clear. 


As with responsiveness, the consist- 
ency data for these comparisons 


-are remarkably uniform and echo the 


results from the illustrative com- 
parison. That is, the two formats 
generate very different pictures of 
our clients, because clients claim 
far more activities, problems, and 
so on.in response to the yes~-no 
format than when asked open-ended 
questions. Consistency for these 
comparisons varied, but in all 
cases the largest source of inconsist- 
encies consisted of "yes" responses 


to the yes-no format coupled with 


Cu 


"not mentioned" responses to the these questions suggests that 
open-ended question. l acquiescence leads mentally re- 

i tarded persons to overreport what- 
ever they are being asked about. 
Open-ended questions have the ad- 
vanta: of relatively high agree- 
men*, “ ‘enerate very low respon- 
Sivene. , “2 that information 
drawn from open-ended questions re- 
presents only a selected: fraction 
.of the population. Further, the 
number of items enumerated in re- 
sponse to these questions is low 
even among those who respond ap- 
_propriately; the very low pro- 
portion of mention-mention agree- 
ments in these comparisons raises 
which the client answered "yes" the possibility of underreporting” 
and the attendant answered "no." by both mentally retarded respon- 
The fact that clients clearly dents and their attendants. The 
acquiesced even in response to these truth probably lies somewhere be- 
questions, for which "yes" responses ` tween these extremes of under- 
were -not socially desirable responseS, reporting and-‘6verreporting. Un- 
indicates that in at least some fortunately, it seems impossible : 


Only one of these comparisons, 
contrasting an open-ended—ques- 
tion asking the client to enumerate 
problems that he or she has with, 
a series of yes-no questions about 
particular problems, included 
attendants' responses. Agreement 
was far lower for the yes-no than 
the open-ended questions (average 
agreement of 45.6% and 92.4%, 
respectively). The yes~-no format 
again led to. overreporting because 
of acquiescence, as 44.9% of com- 
parisons were disagreements in 


cases acquiescence per se can over- to establish exactly where, thdugh 
rule the influence of social de- our findings hint that overreporting 
sirability. The open-ended format in response to yes-no questions ‘may 
again appeared to lead to under- be the more prondéunced bias. 


reporting, as all disagreements 
(7.5% of pairs) occurred when an 
attendant said "yes", regarding a 
problem not mentioned by the client. 
The very high agreement on this 
question resulted from the fact that 
neither clients nor attendants men- 
tioned many problems; in 89.1% 

of pairings, the two agreed in that 
neither indicated a given problem. 


DECISION-MAKING QUESTIONS 
Illustrative Comparison 

- Two open-ended questions, "Who 
decides what chores you do?"\ and 
"who decides what chores to do?” 
were compared with a yes-no ques- 
tion, "Do you decid2 what chores to 
do?" The first open-ended ques- 
tion was asked on both forms, al- 
lowing for three comparisons of 


Conclusions i the open-ended and yes-no formats. 

These comparisons are an excel- These questions differ fundamentally 
lent illustration of the/difficul- from open-ended questions previ- 
ties of interviewing mentally re- -> ously discussed, in that they do , 
tarded individuals, as the use of not solicit enumerations, Rather, 
two different methodologies to ‘the client's response was coded 
seek the same information yielded __ into one of two categories (client 
completely dissimilar pictures of decides vs. someone else decides). 
the target population. Yes-no ques- Thus, the information demanded by 
ticas have the advantage of high these questions is closely compar- 
responsiveness, but are clearly sub- able to that solicited by either- 
ject to strong systematic bias or questions. 


(acquiescence). Low agreement for 
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tion. 


Responsiveness for the three open- 
ended questions ranged from 32.8% 

to 34.5%, averaging 33.9%: This 
figure was clearly exceeded by the 
54.4% responsiveness for the yes- 

nc question; the superiority of the 
yes-no format in eliciting appropri- 
ate responses emerges with monotonous 
regularity. 


Consistency for the three comparisons 


of the yes-no questions with open- 


ended questions ranged from 56.3% 

to 68.9%, averaginy 63.9%. Exam- 
ination of the data reveals a def- 
inite pattern in the inconsistencies. 
Of all comparisons, 32.4% were errors 
in which the client responded "some- 
one else" to the open-ended question, 
but answered "yes" (client decides) 
when asked the yes-no form of the 
question. This fraction accounts 

for nearly all the inconsistencies; 
clearly, acquiescence powerfully in- 
fluenced the responses of clients 

to the yes-no format. Asa result, 
one would conclude on the basis of 
the yes-no question that 52.1% of 

the clients decide for themselves, 
whereas: the three open-ended ques- 
tions yield a much lower figure 
(23.4%). 


Agreement with attendants differed 
radically for the yes-no and open- 


. ended questions; agreement for the 


yes-no question was 34.6%, while 
agreement averaged 77.9% for the 
three open-ended questions. For 
the yes-no question, 65.4% of com- 
parisons were disagreements in which 
the client responded "yes" (client 
decides), while the attendant re- 
sponded that someone else decided. 
Again, acquiescence operated to re- 
duce the validity of responses of 
ea adults to the yes-no ques- 


Additional Comparison 

A similar comparison of yes+no 
and open-ended questions, also 
conducted with institutionalized 


adultS, involved decisions about 
„spending money. The results of 
this comparison are virtually iden- 
tical to those of the previously 
discussed comparison. 


Conclusions 

As always, responsiveness for — 
yes-no questions was far higher 
than for open-ended questions. 
However, in these comparisons the 
reliability and validity of the 
open-ended questions were superior. 
Client-attendant agreement was in 
each case much higher for the open- 
ended questions, as acquiescence 
apparently reduced the validity of 
responses to the yes-no questions. 
It appears that open-ended ques- 
tions soliciting information which 
can be coded into a few present. 
categories may be quite useful, as 
Jong as one is willing to make a 
sacrifice in cesponsiveness. 


Yes-No Vs. Either-Or Questions 

On another test of alternative 
formats, we compared. sets of yes- 
no and either-or questions dealing 
with the same topics. A clear 
comparison of these formats re- - 
quires an understanding of the 
strengths and weaknesses of each 
-questioning strategy taken sep- 
arately. Thus, we begin by analyz- 
ing responses to sets of oppositely 
worded yes-no questions and to sets 
of either-or questions with response 
alternatives presented in reverse 
order. 

f 4 

OPPOSITELY WORDED YES-NO QUESTIONS 

Two comparisons of oppositely 
worded yes-no questions will be 
discussed below. Responsiveness 
is unaffected by reversiny the 
wording of yes-no questions and 
is not analyzed here. 
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Table 8-3: 


i : 
Oppositely Worded Yes-No Questions 









Mes somebody else decide 
how you spend your money? 








‘Illustrative Comparison 


Institutionalized adults were 
asked "Do you decide how to spend 
your money?" and "Does somebody 


. else decide how to spend your 


money?" The contingency table 

for comparison of the two questions 
is depicted in Table 8.3. Only 
38.3% of the adults, far below the 
50% which would be expected if re- 
sponding to both questions were 
random, gave consistent responses. 
Whi7.e 76.5% said "yes" to "Do you 
decide...?" 61.8% also said "yes" 
to "Does somebody else decide...?" 
Fully 50.0% of the response com- | 
parisons were self-contradictions 
in which the client responded "yes" 
to both forms of the guestion. , 
Acquiescence is clearly a major de- 
terminant of responses to these’ 
questions. 


For the "Do you decide...?" ques- 
tion, agreement with’attendants was 
55.5%. For the/oppositely worded 
question, agreement was 54.3%. In 
both cases, the majority of dis- 


agreemants consisted of a "yes" re- 
g 


sponse from the client, paired with 

a "no" response by the attendant, 
again indicating the importance of 
acquiescence: by mentally retarded per- 
sons. Another set of oppositely- 


Do you decide how 


to spend your money? 
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Consistency = 38. 3% 





worded yes-no questions, asked of 
community children, produced vir- 
tually identical D i 


Conclusions 

Both of these pairs of oppositely 
worded yes-no questions clearly il- 
lustrate the enormous importance of 
acquiescence among mentally retarded 
persons. These respondents tend to 
answer "yes," even when such re- 
sponses are self-contradictory. 


Additionally, clients tended to 
respond "yes" far more often. than 
significant others. Clearly,. 
acquiescence effects are powexful 
enough that the validity of reà 
tarded persons' responses to yes- 
no questions is minimal. gs 


REVERSED EITHER-OR QUESTIONS i 
Iilustrative Comparison 

A number of pairs of either-or 
questions were asked, such that 
the members of the pair dealt with 
identical content but the order of 
presentation of response alterna- 
tives was reversed. One such com- 
parison in the children's community 
sample will be discussed below. 
The questions were "Most days, does 
somebody eise tell you what to 
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wrar to school, or do you decide?" 
and "Most days, is it up tọ you 

to decide what to wear to school or 
does somebody else tell you what 

to wear?" 


Responsiveness, 63.6% to the first 
either-or question and 65.5% 

to the second, was unaffected by 
the change in wording. Consistency 
was impressive for this pair of 
questions,” in that 83.3% of chil- 
dren who responded appropriately 
to both questions responded in a 
consistent manner. The type of 
inconsistency was evenly divided, 
suggesting a lack of position 
preference in response to this 
question. 


`” Agreement with parents was fairly 


high for both forms of the question. 


k Agreement was 72.5% for the ques- 


/ 


\ tion in which "somebody else" was 
‘first alternative, and was 73.2% 
for the other question. For the 
former question, types of dis- 
agreement were evenly mixed; for 
the latter, most inconsistencies 
(19.5% of comparisons) involved 
the choice of the last option by 
the child. This constitutes evi- 
dence for a weak poSition bias in 
favor of the last-mentioned alter- 
native. : 


Conclusions 

This pair of either-or questions 
generated relatively high consist- 
ency and parent- child agreement. 
There was some evidence of a weak 
position bias by mentally retarded 
persons, but little indication of 
powerful response effects. To this 
point, then, either-or questions ap- 
pear superior to yes-no questions, 
but lead-to-head comparisons of 
the two formats are needed before 
we can be sure. 


YES-NO VS. EITHER-OR: FOUR-QUESTION 
SETS FROM INSTITUTIONALIZED ADULTS 

‘Two sets of questions asked of 
institutionalized adults, com- 
paring either-or and yes-no ques- 
tions, were repeated in two inter- 
views a week apart, yielding a 
measure of test-retest reliability 
as well as consistency measures. 


Illustrative Comparison 

One of these four-question sets 
inquired about the client's happi- 
ness, using the following questions: 


usually happy? 
usually sad? 

usually happy or sad? 
usually sad or happy? 


Are you 
Are you 
Are you 
Are you 


All questions were repeated on 
bot! administrations of the inter- 
view, so that a total of eight 
questions is involved in this com- 
parison. 


Average responsiveness for the eight 
questions was 65.5%. Responsiveness 
ranged from 58.6% to 70.7%, and 

was slightly higher for the four yes- 
no questions than for the either-or 
questions--68.1% vs. 62.8%. 


One measure of consistency compared 
each question with its identical 
counterpart on the other form, giving 
a measure of test-retest reliability. 
For the "happy" and "sad" yes-no 
questions respectively, consistency 
figures were 97.3% and 60.5%. For 
the two either-or questions, consist- 
ency .figures were 87.9% and 94.1%, 
resulting in average consistency 

for the four comparisons of 85.0%. 


-One point of. interest is that consist- 


ency was very discrepant for the 

two yes-no questions. It seems likely 
‘hat this is an effect of acgquies- 
cence; fcr the "happy" worċing, 
"true" rasponse and their response 
set coeraved together, so that "yes" 
responses were firmly established. 
On the "sad" wording, response bias 
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clients' 
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and the "true" response operated 

in opposite directions. This conflict 
apparently introduced a degree of 
uncertainty which caused some clients 
to change their responses from inter- 
view to interview. Thus, the test- 
retest reliability of the ves-=no 
questions appears to have depended 

in part on the wording of the ques- 
tions. In contrast, consistency 

for the either-or questions dif- 
fered only slightly as a function 

of wording. Test-retest reliability 
for both these types of questions 
was reasonably high. It was some- 
what higher for either-or questicns 
(on the average, 91.0% vs. 78.9%), 
probably because response set was 

of less importance for the either- 

or format. 


Another type of consistency involved 
comparison of oppositely worded 
questions within a given format. 

For the four comparisons of opposite- 
ly worded yes-no questions asked 

on the two occasions, consistency 
averaged only 53.1%, ranging from 
46.2% to 55.6%. Clearly, acquiescence 
was a major determinant of response 
for these questions, as on the aver- 
age, 43.9% of clients contradicted 
themselves by responding "yes" to 
both questions. For comparisons of 


oppositely worded either-or questions, 


Consistency ranged from 77.8% to 
99.7%, averaging 86.2%. This rela- 
tively high, uniform consistency 
attests to the usefulness of either- 
or questions in interviewing mentally 
retarded individuals. However, there 
was some indication of a last option 
Position bias inherent in responses 
to either-or questions. Clients 
tended overwhelmingly to respond 
“happy” to these questions, but this 
tendency was more pronounced when 
"happy" was the last mentioned re- 
sponse alternative (93.4% reported 
weing "happy" when "happy" was -the 
last choice vs. 81.8% when it was 
not). This response bias, however, 


was much less powerful chan was 
the acquiescence in response to 
yes-no questions. 


This set of questions allowed fcr 
sixteen comparisons of yes-no to 
either-or questions. The eight com- 
parisons of the question "Are you 
usually happy?" with either-pr ques- 
tions yielded an average consistency 
figure of. 96.7%, ranging from 82.9% 
to 100.0%. These comparisons tended 
to be very consistent because a 
large majority of clients responded 
"ves" to the yes-no question and ~ 
"happy" to tie either-or format; 

an average of 87.4% of comparisons 
were consistencies or Uris type. 
However, for the eight comparisons. 
of the question "Are you usually 
sad?" with either-or questions, con- 
sistency averaged only 57.3%. For 
this set, an average of 40.0% of 
format comparisons were inconsist- 
encies in which the; client re- 
sponded "yes" to the yes-no ques- 
tion but answered ,“happy" when asked 
the either-or qués’.ion. Clearly, 
“happy" was generally the "true" 
response of mentally retarded 
individuals to these questions, 

and responses to the either-or ques- 
tións tended to reflect that fact 


yo spite of: some order preference). 


However, when the yes-no question 
was worded so as to bring tendencies 
toward acquiescence and the "true" 
response into conflict, acquiescence 
caused many clients to inaccurately 
report that they are usually sad. 
The striking difference between . 
the 87.4% and 57.3% average consist- 
ency figures for comparisons of 
either-or questions with the two 
yes-no questions attests to that 
fact. i 


Average agreement with attendants 

for the either-or questions was 

85.8% and ranged from 79.3% to 94.2.. 
Average agreement for the yes-no 
questions was somewhat lower at 72.1%. 
This reduction in agreement is ac- 
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counted for by the very low agree- 
ment for the yes-no question "Are 
you usually sad?" which was 50.0% 
and 50.7% on the two forms. For 
this question, an average of 43.7% 
of client-attendant pairs were 
inconsistent in that the client 
said "yes" (sad) and the attendant 
indicated that the client was usu- 
ally Happy. Again, the major con- 
clusion is that acquiescence largely 
j A the responses of men- 
tally retarded respondents to yes- 


no questions. 
Wa 


Additional Comparison g 

Another comparison of yes-no 
and either-or questions, including 
measures of test-retest reliability, 
was conducted with institutionalized 
adults. This sét of questions asked 
whether clients usually spend their 
‘time alone or with othex_people..... 
Average responsiveness..for the 
eight questions was 63.2%, and was 
slightly superior for the yes-no 
questions (68.3% vs. 58.0%). Test- 
retest reliability for the set av- 
eraged 76.2%. As in the previous 
set, the reliability of yes-no ques- 
tions was augmented when acquies- 
cence set and the clients' "true" 
answer coincided and was diminished 
when they were at odds. 


Response consistency was again abys- 
mally low when the oppositely worded 
yes-no questions were compared, av-' 
eraging 52.0%. The most frequent 
response combination (43.4% of com- 
parisons) had-clients contradicting 
themselves by responding "yes" to 
both questions. Comparisons..between 
oppositely worded either-or ques- 
tions averaged 75.1%, suggesting 
superior validity..for either-or 
questions. Again last option re- 
sponse bias was dvServed, as the 

‘ most frequent inconsistent response 
combination (18.3% of all compari- 
sons) involved the client choosing 
the last option both times, as op- 
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posed to only 6.6% of clients who 
chose the first option both times. 


_ Comparisons of yes-no and either- 


or questions did not reveal the rad- 


ical differences in ccnsistency fig- 


ures for the two yes-no questions 
that characterized the "happy-sad" 
set. For the eight comparisons of 
either-or questions with "Are you 
usually with other people?" con-- 

sistency averaged 69.5%. For the 

question "Are you usually by your- 
self?" consistency averaged 68.7%. 


“This probably reflects the fact that 


answers were much more balanced 
between the two response alternatives 
for the alone-with others questions 
than for the happy-sad questions. 


Average agreement for ihis set was 
60.8% for the yes-no questions and 
60.3% for the either-or questions. 
Again, agreement figures for the 
two-yes“no-questions were discrepant, 
averaging 51.4% for "Are you usually 
by yourself?" and averaging 70.2% 
for "Are you usually with other 
people?" For the either-or questions, 
agreement was slightly higher for 
the question for which the most 
frequently chosen response alterna- 
tive was the last mentioned, 63.5% 
as opposed to 55.2%. Thus, client a 
attendant agreement is an over- 
estimaté of response validity when 
the "true" response and client 
response bias happen to coincide- 
This effect is apparent in the 
either-or comparisons, where agree- 
ment is greater when the "true" 
answer (with others) is the last- 
mentioned alternative. However, 

it is far more notable for the ys- 
no questions, where agreement is 
much higher when "yes" indicates 
that che client is usually with 
other people. 


Conclusions 

Both sets of questions suggest 
that either-or questions compare 
favorably with yes-no questions 
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as a method of gaining information 
from mentally retarded persons. 
Both either-or and yes-no questions 
involve some response biases which 
reduce the validity of responses. 
However, the acquiescence activated 
by yes-no questions is a crippling 
deficiency, whereas the position 
bias inherent in the either-or 
format is merely annoying. In the. 
second set of questions discussed, 
the advantage of the eitner-or 
format is not so great. Neverthe- 
less, consistency for the either- 
or questions in both cases approaches 
the test-retest reliability for 
these questions, and even in the 
second case, either-or questions 
are less affected by response set. 


SUPPLEMENTARY YES-NO 
VS. EITHER-OR COMPARISONS 

The set of yes-no versus either- 
or comparisons described above con- 
stitutes only a fraction of the 
data we collécted regarding the | 
relative merits of these two formats. 
Four separate four-question sets, 
similar to the above examples, were 
included in interviews with insti- 
tutionalized children. In addition, 
two direct comparisons of isolated 
either-or and yes-no questions were 
conducted in interviews with insti- 
tutionalized adults. Although some 
variation in patterns of response 
to thes2 two formats was of course 
observed, the consistency with which 
these data characterized yes-no and 
either-or questions was remarkable. 
Specifically, in every case the 
responsiveness data slightly favored 
the yes-no format. However, in every 
comparison the evidence regarding - 
validity unequivocally recommended 
the either-or questions, usually 
because of the powerful effect of, 


acquiescence on the yes-no format. 


Again we must conclude that there 
is no point in eliciting invalid 
responses; responsiveness must yield 
to validity as a more essential cri- 


terion. Direct comparisons of yes- 
no and either-or questions suggest 
that the either-or format is pref- 


erable as a means of gaining in= 


formation from mentally retarded 
persons. 


The Use of Pictures With Yes-No 


Questions 

In a continuing effort to maxi- 
mize the amount of information that 
we could gain from clients with low 
verbal skills, we asked some yes- 
no questions which included pictures 
to clarify the question. Two sets 
of questions directly compared yes-no 
questions using pictures with the 
same yes-no questions not using pic- 
tures. These comparisons can tell 
us whether the use of pictures in 
any way serves to reduce the acquies- 
cence so commonly observed in re- 
sponse to yes-no questions. 


Illustrative Comparison 

The first such set, asked of 
institutionalized children, involved 
indoor games. The children were 
twice asked four yes-no questions 
such as "Do you Play checkers?" 
once with words only and oyce in- 
cluding pictures of people engaged 


in the game named. 


Responsiveness, 84.4% for the ques- 
tion with words only and 82.2% for 


‘the words plus pictures format, 


was apparently unaffected by ques- 
tion type. Respondent consistency 
for the four pairs of yes-no ques- 
tions averaged 89.8%. Children 
tended overwhelmingly to: respond 
"yes," with the result that most 
comparisons (75.1%) were consist- 
encies in which the child responded 
"yes" both times. 


The agreement data contain the most 
useful information regarding this 
format comparison. These data are 
summarized in Table 8.4. Overall 
agreement averaged only 42.4%. By 
far the greatest fraction of dis- 
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table 8.4; Yes-No Questions with Words Plus Pictures vs. Words Only - 


— amamaaen a a 








Child-No 


tion T f i N 
Questi ype opic S.O. = Yes 


Checkers 45 
No pictures 

Cards 44 

Dominoes 43 


Other games 4l 


p 


Group Average 43.3 47.4 38,6. 8.7 42,1 10.6 





Checkers | l 2 l 34.9 2.3 

Pictures 
Cards i : ; 38.1 4,8 
Dominoes ; 55,8 11.6 
` Other games |38 n i ; 34.2 | Ld 
Group Average 40.8 10.0 
Overal! Average 42.4 48,3 38.9 9.5 41.4 10.3 
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agreements consisted of "yes" re- 
sponses by clients coupled with 

"no" responses by attendants (41.1%, 
on the average), indicating power- 
ful acquiescence effects. More 
important is the observation that 
the two formats differ little; they 
are characterized by equally low 
agreement and equally strong re- 
sponse biases. 


Additional Compa.cison 

A similar comparison of yes- 
no questions using pictures was 
conducted in the children's insti- 
tutional sample, where children 
were asked if they engaged in.ten 
different chores. Responsiveness 
averaged 69.8% for the ten pictures 
questions, vs. 70.9% for the words 
only format. Consistency of respond- 
ing between formats, averaged across 
the ten questions, was 87.9%. This 
consistency tended overwhelmingly 
to result from the client answering 
"yes" to both formats; 74.1% of all 
comparisons were consistencies of 
this type. For the picture format, 
client-attendant agreement averaged 
72.1%, resulting mostly from cases 
in which both client and attendant 
answered "yes." Most disagreements, 


‘20.4% of all comparisons, occurred 


because the client responded "yes" 
and the attendant responded "no," 
indicating acquiescence. This pattern 


of responses was virtually repeated 


in the words only format, where over- 
all agreement was 69.5%. 


Conclusions 

Yes-no questions using pictures 
tend to be invalidated by acquies- 
cence, as are similar questions using 


words only. .The use of pictures does ` 


not increase responsiveness or in 
any way improve the information ob- 
tained using yes-no questions. In 
short, the use of pictures offers 
no advantage over the simple words 
a format. oo 


The Limits of Acquiescence 

By now, the reader may be weary 
of our continual references to the 
problem of acquiescence. Yet it is 
of such importance that we feel a 
need to draw together some of our 
findings and present still more so 
that the implications of acquiescence 
are clearly appreciated. 


One of the pairs of oppositely worded 
questions that were used in all three 
major samples was "Are you usually 
happy?" versus "Are you usually sad?" 
Of those persons who could respond 

to both questions, fully 50.9% of 

the community children, 43.8% of - 
the institutionalized children, and 
41.7% of the institution adults ac- 
quiesced, that is, answered "yes" 

to both questions. The correlations 
between acquiescence to this pair 

of questions and IQ in these samples 
were -.30, --40, and -.41, respec- 
tively, all significant at the .05 
level and all suggesting that lower 
IQ respondents are more likely than 
higher IQ respondents to acquiesce. 
The oppositely worded questions 

“Are you usually by yourself?" and 


"Are you usually with other people?" 


were also used with all, three samples. 
Here, 50.0% of the community children, 
39.4% of the institution children, 
and” 41.7% of the institution adults 
contradicted themselves by saying 
"ves" to both questions. The correla- 
tions between acquiescence and IQ 
were -.24, .03, and ~.04, respective- 
ly, with only the first statistically 
significant. Thus, in all three 
samples. and on both pairs of ques- 
tions, acquiescence rates were alarm- 
ingly high. And although lower IQ 
respondents tended to acquiesce 

more, the correlations were not 

large enough or stable enough to 
suggest that severely retarded per- 
sons are the only culprits. Instead, 
acquiescence was a pronounced prob- 
lem of all levels of retardation. 
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Still, having found some relationship 


between acquiescence and IQ, we 
explored the matter further in new 
interviews with 29 moderately and 
severely retarded institutionalized 
children and adults who had proven 


and "alone-with other" questions 
as well as other oppositely worded 
pairs, some of which were factual 
rather than subjective in nature. 
We/ also used a second approach to 
“smoking out" acquiescers, asking 


questions that demand a "no" answer. 
The results we obtained are pre- 
sented in Table 8.5,.A ani B, and 


to be largely, but not totally, unre- 
sponsive in their original inter- 
views. We included the "happy-sad" 


a Imm amamammmmmmmaaaauamummaaumumamamummmmmħį 


Table 8-5A: Rates of Acquiescence in Response to Pairs 


of Oppositely-Worded Questions 


a a aaam aaa aumum 


Percent 


Questions Acquiescing 


Responding 





l. Are you usually happy?" 
usually sad? 


vs. Are you 23 27.8% 


— 








Qe. Are you usually with other people? 25 56.0% 


vs. Are you usually alone? 


3. Most mornings here, do you pick ` 21 
out what clothes to wear? vs. 
Most mornings here, does somebody 
else tell you what clothes to wear? 


71.4% 


4. Do you live.in’G-10 (incorrect 25 
cottage number) right now? vs. 
Do you live in G- (client's 
cottage number) right now? 


20.0% 


5. Here is a picture of (super- 24 

l intendent's name). Is he the 
superintendent here at the 
colony? vs.. Here is a picture 
of (business manager's name). 
Is he the superintendent here 
at the colony? 


83.3% 


Average: 55.7% 
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they must be seen to be believed. 
Unless answers to yes~no questions 
were viewed with a jaundiced eye, 

one might conclude that a large 
proportion .... this sample consists 

of Chinese schooi bus drivers who 

fly airplanes on the side! It is 
important to note, however, that 
acquiescence rates varied from ques- 
tion to question. For example, al- 
though these respondents demonstrated 
an almost total lack of knowledge 

of the identity cf the superintend- 
ent of the institution and acquiesced 
in response to the pair of questions 
about this topic over 80% of the 
time, they apparently were more cer- 
tain of their cottage numbers anc 
only 20% acquiesced when asked if 
they lived in another cottage. 


To explore tie implications of these 
staggering rates of acquiescence, 

we examined the extent to which these 
same respondents claimed to be in-. 
volved in four chores in the cottage. 


The average respondent claimed to | 


be involved in 80% of the chores; | 
yet when we asked attendants the 
same questions, they claimed the 
average resident was involved in 
only 59% of the chores. More im> 
portantly, those respondents who 
showed themselves to be especially 
prone to acquiesce tended to claim 


‘more involvement in chores. If we 


had relied uncritically on these 
answers, we would have concluded, 
on the basis of a correlation of 


-.61, that lower IQ respondents 


were more involved in chores than 
































Tabie 8.5B: In Response to Questions for which 
"No" is the Correct Answer 
Question Responding % Yes Response 
l. Are you Chinese? ae 25 44.0% 
a l 
2. Does it ever snow here 26 73.1% 
in the summer? 

‘3. Do you know how to fly an airplane? 25 52.0% 
4. Are you a school bus driver? 24 41.7% 
5. Right now, is it raining outside? 25 7 l 28.0% 

(or sunny outside, whichever question i 7 
should be answered "no" at the time , 
of the interview) 
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higher IQ respondents. Yet this . 
finding would have been an arti- 
fact, for lower IQ respondents also. 
scored higher on our measures of 
tendency to acquiesce. Using attend- 
ant data in place of client data, 

the correlation between IQ and in- 
volvement in chores turned out to 

be .26 rather than -.61. Thus this 
-analysis not only confirmed the 
finding that acquiescence tends to 
be more likely among the more severely 
retarded but also suggested that 
their acquiescence can distort the 
relationship between IQ and the 
topics of interest in an interview 
(in this case, involvement in 
chores). We trust that these addi- 
tional analyses leave no doubt 

about the magnitude of the problem of 
acquiescence. Responses to yes- 

no questions simply cannot be trusted. 


The Use of Pictures r 


With Either-Or Questions. 


‘In a continuing effort to maxi- 
mize responsiveness while minimizing 
systematic bias, we tested some 
either-or questions which offered 
pictures as response alternatives, 
hoping to increase responsiveness 
to either-or questions without de- 
creasing their generally high 
validity. 


QUESTIONS ASKED OF COMMUNITY CHILDREN 


Illustrative Comparison 

Community children were asked 
whether they usually spend their 
time alone or with others, using 
the following set of four questions: 


When you're not at school,are ` 
you usually by yourself or with 
other people? | í 
‘When you're not at school, are 
you usually with other people 
or by yourself? 
This picture shows a boy/girl 
who is by himself/herself and 

- í 


; 
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this one shows a boy/girl who 
is with other people. When 
you're not at school, which 
boy/girl is most like you? 
Point to the picture. 

This picture . . .(same.as 
above with order of response 
options reversed) 


Data collected from these four ques- 
tions are summarized in Table 8.6. 


As expected, the children were con- 
siderably more responsive to the’ 
two picture choice questions than to 
the verbal either-or questions. For 


‘the set of four questions, tI:2re are 


six unique pairs, allowing for six 
consistency figures. One consistency 


- comparison is a within-format com- 


parison between the two verbal either- 
or questions. The consistency figure 
for that comparison was only 58.1%. 
The largest fraction of inconsistencies 
for this comparison (25.6% of response 
pairs) occurred because the client 
chose the last option in both cases, 
suggesting a last option position 
preference. ` . 


A second consistency comparison was“ 
between the two picture choice ques- 
tions; Consistency for this comparison 
was a respectable 72.7%. Inconsist- 
encies were nearly equally balanced, 
with a slightly larger proportion 

of inconsistencies due to choice 

of the first option in both cases’ 
(14.7% of comparisons), suggesting 
that response bias may be lessened 
or possibly reversed in the picture 
choice format. The high consistency 
of these auestions casts picture 
either-or questions in a favorable 
light. 


~ 


š aa 
The remaining four consistency com- ` 


‘parisons were between, picture choice 


and verbal eithe*s-or formats. These.: 
consistency figures ranged from 

55.1% to 63.0%, averaging 59.3%., 

The patterns of inconsistencies among 
these questions were not particularly’ 
revealing. 


Table 8.6: Either-Or vs. Either-Or with Pictures 


Question | Responsiveness Mean Consistency* | % Agreement 


When you're not at school, are you usually by 69,68 57,9% 51.08 
yourself or with other peoole? 


| 


When you're not at school, are you usually 80.08 ` | 39, 3% 50.08 
with other people or by yourself? ! 


This picture shows a boy/girl who is by- 
himself/herself and this one shows a 
boy/girl who is with other people. When 
you're not at school, which boy/girl! is 
most like you? Point to the picture. 













94.7% o BA ALS 





This picture shows a boy/girl who is with | 94.7% i 62,28 50.08 
other people, and this one shows a 
boy/girl who is by himself/herself. | 
When you're not at school, which | 
boy/girl is most like you? ` | 
Point to the picture. 







Average: 6).28 48.1% 
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. Average parent-child agreement for 


the four questions was only 48.1%, 
which was below the level of agree- 
ment expected if both parents and 


children selected their responses 


randomiy. Further, the two formats 
differed little in terms of this 
estimate of validity. Agreement 

for the verbal either-or questions 
averaged 50.5%, as opposed to 45.8% 
for the picture choice questions. 
Thus, the major conclusion suggested 
by the agreement data is that neither 


_format elicited substantial agree- 


ment with parents on this topic. 
Still, the findings point to the 
effectiveness. f pictures as a sup- 


_plement to these either-or questions, 


for they increased responsiveness 
and reduced a bias toward selecting © 
the last option given. l 


Additional Compärison 

A_ similar set of four questions, 
systematically varying both format 
and order of response options, was 
asked of community children. These 
questions asked if the child is 
usually happy or sad. Responsive- 
ness to the two picture either- 
or questions averaged 97.4%, ex- 
ceeding the responsiveness figure 
for the two verbal either-or ques- 
tions (90.0%). Consistency for the 
comparison of the two verbal either- 
or questions was 75.5%. Most in- 
consistencies (15.1% of compari- 
sons) involved choice of the first 
option in response to both questions, 
demonstrating that last-option bias 
for these questions is not a very 
robust phenomenon. Within-format 
consistency for the two picture 


- choice questions was 67.8%, with 


inconsistencies more or less evenly 
mixed. ‘For the four comparisons 

of picture arid verbal either-or 
questions, consistency averaged 
72.3%, with no pattern readily ap- 
parent in the inconsistencies. Av- 
erage agreement between parents 

and children for the verbal questions 


was 79.0%, as opposed to 70.4% for 
the picture either-or questions. 


Conclusions 
Both comparisons indicate that 


. the use of pictures as response 


alternatives maximized responsive- 
ness to either-or questions, a 
finding consistent with nearly all 
data previously analyzed. As regards 
the questions of reliability and 
validity, no clear conclusions can 
be drawn from these comparisons, 
which generated incorsistent re- 


sults. The first comparison suggested 


a reliability advantage forthe pic- 
ture choice format, while the second 
comparison contradicted that result. 
Agreement data were’ also inconsistent. 
Agreement for the alone-with others 
comparison was so abysmally low. that 


. both either-or formats appear inade~- 


quate. For the happy-sad comparison, 
both formats appear in a favorable 
light, with verbal questions gaining 
a slight (79.0% vs. 70.4%) advantage. 
Overall, pictures clearly increase 
the ability of retarded persons to 
answer either-or questions. Since 
pictures do not appear to system- 
“atically threaten validity, their 
use can be recommended. 


INTERVIEWS WITH SEVERELY 
RETARDED PERSONS 

Alternative format comparisons 
to ,this point have left us- trapped 
by a dilemma; namely, the very ques- 


. tions that many severely retarded 


persons can answer, particularly 
yes-no questions, yiéld invalid 
answers. It appeared that picture 
choice either-or questions offered ' 
a last hope of obtaining valid re- 
sponses from retarded persons with 
extremely. limited verbal skills. 
For this reason, we made a final 
attempt to test the effectiveness 
of picture either-or questions as 

a means-of gaining information from. 
severely retarde. individuals (chil- 
dren and adults in the institution 


8.37 


192 


Q 
ERIC 


who had been relatively unresponsive 
when they were first interviewed). 


Illustrative Comparison 

This comparison was identical 
to the happy-sad comparison conducted 
in the community children sample, 
pitting a verbal either-or question 
and its oppositely worded counter- 
part against a picture choice either- 
or question and its complement. ` 


X 


Responsiveness data for this com- 
parison are represented in Table 8.7. 
Once again, the use of the pictures: 


Table 8.7: Verbal and Picture Choice Either-Or Questions asked 


as response alternatives maximized 
responsiveness. Comparison of the 


two verbal either-or questions 
revealed consistency of 71.5%. 


Moreover, this comparison suggested 


that last option response bias 


Played an important role, as all 
inconsistencies (28.6% of compari- 
sons) involved choice of the last 
option both times. Within-format 
consistency for the picture either- 
or questions was comparable (73.0%) 
but, importantly, gave no indica- 


tion of systematic bias. Of the 


seven respondents who gave incon- 


of Severely Retarded Persons 


Question 


Are you usually happy or sad? 


Are you usually sad or happy? 


Format Average: 69.0% 


This picture shows a boy/girl who is 


happy, and this one shows a boy/girl 
who is sad. Which picture shows how 
you usually feel? 


This picture shows a boy/girl who is 
happy, and this one shows a boy/girl 
who is sad. Which picture shows how 
you usually feel? 





Format Average: 


Overall Average: 





Responsiveness 
{4 





tree. —— 


% Agreement 
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sistent responses to the two ques- 
tions, four did so by selecting 

the first option both times, as 
opposed to three who chose the last 


option both times. Thus, consist- 


ency data suggest that the two 
formats are comparable with regard 
to reliability, but that picture 
choice questions have an advantage 
in that they are not affected by 
strong response biases. 


Aqreement data are also summarized 
in Table 8.7. An apparent advantage 
exists for the verbal questions, 

as agreement for those questions 
averages 79.8% as compared with 
69.9% for the picture questions. 
However, agreement figures for the 
two verbal questions were extremely 
discrepant. Although attendants 
consistently responded "happy," 
clients' responses changed as a 


function of the order of response = 


alternatives. Thus, a high agree- 
ment figure (96.0%) for the ques- 
tion "Are you usually sad or happy?" 
represents not response validity 

but an accidental correspondence 

of client response bias with attend- 
ants' response preference. This 

is evident because agreement for 

the oppositely worded questions was 
only 63.6% and most disagreements 
involved choosing the last option 
("sad") while attendants said "happy." 
By comparison, agreement for the 

two picture choice questions was 
relatively consistent and not power- 
fully affected by systematic bias; 
thús it can be concluded that the 
picture choice questions generate 
reasonably valid responses and are 
less distorted by response bias than 
the verbal choice questions. 


Additional Comparisons 

Two additional comparisons eval- 
uated picture chdice either-or ques- 
tions used in interviews with severely 
Yetarded persons. The first comparison 
was identical in form to the above 


cowparison, pitting two pictorial 
against two verbal either-or questions, 
asking in each case whether the client 
is usually alone or with other people. 
As always, the advantage in eliciting 
appropriate responses accrued to 

the picture choice question, by a 
margin of 81.0% to 55.2%. Within- 
format consistency for the two ver- 
bal choice questions was 62.6%. Fully 
25.0% of comparisons were inconsis- 
tencies in which the client chose 

the last option both times, indicating 
the importance of last option response 
bias in this severely retarded group. 
The within-format comparison for 

the picture choice questions yielded 

a higher consistency figure of 88.0%, 
with no evidence of order effects. 
Agreement for these questions was 
quite low, averaging only 58.3% 

for the verbal questions and 53.8% 

for the picture choice questions. 


‘Analysis of the inconsistencies re- 
! vealed that they were in.each case 


almost perfectly split between first 
and last choice responses by clients. 
Cn both verbaland pictorial questions, 
clients were much more inclined 

than attendants to respond "alone," 
implying that this is a genuine 
difference of opinion between clients 
and attendants which is not a matter 
of response set. Whether this dis- 
agreement involves invalid responding 
by clients, a lack of accurate in- 
formation on the part of attendants, 
or is simply a real disagreement 

is impossible to detemine. 


The second compariscn compared two 
picture choice either-or questions 
with three yes-no questions asking 
whether the client or someone else 
usually picks the client's clothes. 
The pictorial either-or clothes 


_ questions yielded an average re- 


sponsiveness figure of 82.8%, which 
exceeded average responsiveness 

for the yes-no questions (67.8%). 
The comparison between the two 
picture-choice questions yielded 

a consistency figure of 62.5%. In 
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thic case, 29.2% of the response 
pairings involved choice of the 
first option both times, suggesting 
that a first.option response bias 
may sometimes be involved in picture 
choice questions. Analysis of agree- 
ment yielded results similar to 
those from the above comparison; 
agreement was only 52.9%, and in 
both questions clients were-much > 
more likely than attendants to assert 
that they. chose their own clothes. 


Conclusions 

The one unequivocal conclusion 
Which can be drawn from comparisons 
of verbal and picture choice either- 
or questions is that picture choice 
questions elicit appropriate re- 
sponses from a larger fraction of 
the population. Many of these com- 
parisons also suggest impressive 
reliability of responses to either- \ 
or questions, and particularly to 
picture choice either-or questions, 
which are sometimes less affected 
by response'bias than are their ver- 
bal counterparts. These data indicate 
that the use of pictures does not 
reduce validity, although in some 
cases it introduces a first option 
response bias which parallels the 
last option bias found on verbal 
either-or questions. Overall, picture | 
choice questions appear to hold promise 
greater than that of any other tech- 
nique we have used as'a method of 
gaining information from more severely 
retarded persons. 


Conclusions 

This chapter has examined the 
advantages and disadvantages of al- 
ternative approaches to obtaining 
information from mentally retarded 
persons. Viewed in conjunction with 
data presented in earlier chapters, 
it enables us to formulate a set 
of empirically-based recommendations 
to follow if one wants to obtain 
valid answers from mentally retarded 
interviewees. 


Broadly, our findings establish that 
what one hears from mentally retarded 


_ persons depends in part on how ques- 


tions are asked. We have demonstrated 
that not only question format but 
question wording and whether or not 
one uses probes after a question 

is asked can have dramatic effects 

on the answers that are given. As 

a result, the central recommendation 
of this study is as follows: Whenever 
mentally retarded persons are to 

be interviewed, there must be careful 
attention to methodological issues, 
particularly question design. The 
reader who has followed our narrative 
this far can scarcely be Surprised 
and may even consider the broad re- 
commendation so basic that it does 
not need stating. Yet, despite the 
fact that many studies relying on 
interviews with retarded persons © 
have been conducted, this is the 
first time, to our knowledge, that 
this crucially. important maxim has 
been stated. It is certainly the 
first time that measurement issues 

in interviewing retarded persons 

have been systematically studied. 

It is now time to end the tradition 
in the mental retardation field 

of simply assuming that responses 

to interviews are valid. 


Based on the findings we have pre- 
sented, we are able to offer the 
following more specific generaliza- 
tions and recommendations. 


1. Question wording matters. For 
example, if a question calling for 
an enumeration of activities in 

å given category (e.g., sports) 
mentions examples (e.g., football 
and baseball), retarded persons 
will be more likely than usual to 
name the very activities used as 
examples. Thus this strategy for 
making a question more concrete 
is counteryroductive and should 
not be used. In addition, presum- 
ably small changes in question 
wording can have major effects on 
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response patterns. For instance, 

‘we found that retarded persons were 
far more likely to say certain un- 
acceptable behaviors (e.g., hitting 
others) were prohibited if the ques- 
tion was phrased, "Are you allowed 
to. . ." than if the question was 
phrased "is`'it against the rules 
to. . ." We even found, thanks to 

a typographical error in the inter- 
view schedule, that responses to. 
“Who decides what chores you. do?" 
and "Who decides what chores to 
do?" differed. In addition, we found 
evidence that retarded persons do 
not always pick up distinctions 

in wording that the researchers 
hope they will; for example, dis- 
tinctions among quantitative words 
such as "always," "usually," " some- 
times," and "never." Finally, we 


found abundant evidence that responses 
to oppositely worded questions (e.g., 
"Are you usually with other people?" 
vs. "Are you usually alone?") can 
differ drastically. There are many 
examples that we can cite of mis- 
interpretations, often humorous ones, . 
of words and phrases in questions 

{see Sample 8.4). As we are not cer- 
tain of all the possible effects 

of question wording, we can only 
recommend that there be extensive 
piloting of interview schedules to 
determine whether respondents under- 
stand and, if so, how they interpret ' 
alternatively phrased questions. 


2. ‘Techniques of probing for addition- 
al ‘information must be carefully 
planned. As we have shown here, when 
interviewees are asked open-ended 
questions calling for an enumeration 
of activities, many cannot respond 

at all and those who can are likely 
to name very few activities. On the 
other hand, the strategy of probing 
indefinitely with "What else?" solves 
that problem but creates another 

that is more serious: ove®Breporting 
of activities in response to the 
implied demand to do so to the ex- 
tent that. answers lose validity. 


Thus, if such probes are used, and 
perhaps they should be to overcome 
the problem of underreporting on 
open-ended questions, they should 
probably be used only once or twice, 
not ad infinitum. Moreover, probes 

or follow-up questions should be 
phrased to hark back to the content 
of the original question (e.g., in- 
stead of "What else?" "What else 

rdo you do at your job?"). This strat- 
egy of reminding interviewees of 

the question content increases 

the ability of interviewees to answer, 
without reducing validity-of\ response. 


More generally, all~of our findings on 
question wording and format need to be 
applied-when probes or follow-up 
questions are used. Probes and follow- 
up| questions should, as a general 
rule, be as carefully planned as 

the’ original questions they follow. 
LeaVing the design of such questions 
to interviewers who are insensitive 

to the implications of question 
structure and wording is essentially 
leaving matters to chance. If one 
interviewer chooses to follow up 

an unclear answer with a simple 

yes-no questions while another chooses 
to use an either-or question to, 
pursue the matter, we can predict 

that the two interviewers are likely 
to get different answers. This is 

by no means a trivial issue, for 

our experience suggests that no 

matter how well ‘original questions 

are designed, there. is tikely to 

be some need to’ follow-up or clarify 
vague or confusing answers.- 


3. Multiple choice questions that 
call for responses along a quanti- 
tative dimension (e.g., a lot, some, 
not much, and-not at all) appear 

to be generally useless as sources 
of information. They are relative- 
ly difficult for retarded persons 
to answer, in the first place; but 
more importantly, respondents appear 
to have difficulty attaching con- 
sistent meanings to’ such terms. 

We did not find any advantage in 
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Samle 8.4 


A Sampling of Unusual Interpretations of Questions 


1. 


Q: 


Which picture shows how you like the food here? 


Foint to the picture. 


-I know, keep your mouth closed. 


Do you decide what chores you do? . 
Probably clean house and fix your bed and clean up hallway 
and mop and go walking and go see somebody's house. 


— 


Are you allowed to hit people? ' 


I don't hit people. 


What do you and your friends usually do together? 
Let's see. My mom says.. (Puts hand to head as if in 
deep thought). Let i think. Well, I say yes. 

Do people here help you when you want help? 

No, I help myself to bed, go to the bathroom. 

If you had one wish, what would you wish for? 

Well, ras to seel and we're going to have a party this. 
afternoon. 

Do ai decide what chores you do? 

We go to the canteen and get something. 

Do you decide what chores you do? 

Not until I'm Assigned to it. 

Do people here take things ¿way from you? 


No, but they do steal. 
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10. Q: Do you dust furniture? 
A: No, wash ears, that's all. 
1l. Q: Are you allowed to go on dates here? 
A: No, my daddy is. 
12. Q: Has anybody from Voc. Rehab. ever tried to help you? 
| A: (nodded yes) 
Q: How did they help you? 
A: Wash your head. 
13. Q: Where do you get your money? 
A: My money, in my e. 
14. Q: Do people here make you do things you don't want to do? 
| A: It's not wise. 
15. Q: Could you buy some candy with $1.00 (question designed to 
check eT ee 
A: I done it, but I had pimples once before. 
Q: (repeated) 
A: If I don't get a hot dog. 
“a6: .Q: Do you weudiiy have problems knowing what to do when you 


get sick? 


A: No, always run to the bathroom and get sick. 
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breaking up a four-choice multiple 
choice question into a yes-no ques- 
tion and then a muitiple choice 
question (e.g., "Do you leave here 
to go see your family?" if yes "Do 
you go see them a lot, sometimes, 
or not much?"). Nor did we find ` 
smiling and sad faces as an alter- 
native to verbal multiple’ questions 


about satisfaction useful, except 


to’ boost responsiveness. ; 


Unfortunately, we have no con- 
structive advice for interviewers 
who seek quantitative information 
about such things as extent of 
involvement in various activities. 
Based on previous experience in 
other research projects with ques- 
tions calling for estimated numbers 
of times doing something in a given 
time period, we steered away from 
that approach. We simply do not feel 
that retarded persons typically have 
a good enough grasp of quantitative 
concepts and time concepts that they 
can respond validly. 


4. Multiple choice questions offer- 
ing discrete response alternatives, 
however, may have promise. In asking 
respondents what type of dwelling 
they lived in and how they got to 
school, we found that questions of 
this type worked well; better than 
cpen-ended questions on the same 
topics, for they enhanced responsive- 
ness without sacrificing validity. 
Although responses to verbal multi- 
ple choice questions were sometimes 
more affected by the order of options 
than were responses to parallel pic- 
ture choice questions, we are not 
confident that pictures provide 
enough of an advantage to warrant 

the effort involved in producing 
them. Moreover, we are not able to 
say whether multiple choice questions 
naming discrete response options would 
work in other content areas, as we 
used them only to get information 
about basic, and presumably well- 
known, factual information. 
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5. Yes-no questions, despite the fact 
that they are easily answered by most 
retarded persons,’ are not useful in 
interviewing retarded persons. While 
this statement may seem strong, we 
have devoted more attention to yes-no 
questions than to any other format 

and we have found, with few exceptions, 
that retarded pérsons are highly like- 
ly to answer "yes" regardless of ques- 
tion content. Some of our evidence of 
acauiescence in response to yes-no 
questions cannot be described as any- 
thing other than shocking. In compari- 
sons of oppositely worded yes-no ques~ 
tions, it was not uncommon for the most 
frequent pairing of responses to the 
two questions to be "yes" to both 
rather than the expected "yes" to 

one and "no" to the other. We have 
witnessed severely retarded persons, 
asked questions that should be an- 
swered "no," claim through their 


` yeasaying, to be Chinese school 


bus drivers who fly planes on the 
side. The impact of acquiescence 

is so strong that we feel yes-no 
questions should be To entirely. 
If they are used, we can see no 
alternative but to build into the 
interview some check on acquies- 
cence (see Chapter 9 for some sug- 
gested strategies). 


6. Open-ended questions have one 
overriding limitation: they cannot 
be answered by very many retarded 
persons. As a result, they are not 
very useful at all in interviewing 
More severely retarded persons. 
However, they can be of use in inter- 
viewing persons with relatively , 
advanced verbal skills. We found |, 
that simple open-ended questions / 

in which a single response can be’ 
coded into categories (e.g., "Who 
decides’what chores you do?") yielded 
relatively valid answers from those 
who could respond. Open-ended ques- 
tions calling for enumerations (e.g., 
"What things do people here do that 
bother you?") were less satisfactory. 
In comparisons of them with parallel 
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sets of yes-no questions, we found: 
that they probably led to underre- 
porting, although it was not so 
pronounced as the overreporting 

in response to yes-no questions. 

Very few retarded persons appear - ~ 
to have the level of verbal skill 
that such enumeration requires. 

There is oft@n no choice but to 

ask some open-ended questions in 
interviews. As long as the respondents 
are verbally skilled, these questions 
can yield great insight into the 
lives of retarded persons. The only 
caveat is that the insights gained 
Will probably not be representative 
of the larger population of persons 
one is studying. Thus we can best 
envision open-ended questions being 
used as follow-up questions to more 
structured questions, to gain addi- 
tional clarification of answers. to 
those structured questions from those 
capable of providing it. 


7. Of all the question formats we 
have tested, either-or questions, 
particularly those accompanied by 
pictorial representations of the 
alternatives, appear-to be the best 
way of optimizing responsiveness 
and response validity. In numerous 


‘comparisons of either-or and yes- 


no questions, either-or questions 
consistently emerged victorious. 


While they are somewhat more diffi- 


cult to answer than yes-no questions, 
this slight sacrifice in responsive- 
ness is more than made up for by 

a gain in response validity. We have 
found some evidence of a preference 
for the last of the two options pre- 
sented, but it has not been totally 
consistent and, more importantly, 

it has typically been a minor re- 
sponse bias in comparison to acquies- 
cence. Furthermore, the use of pic- 
tures as adjuncts to either-or ques- 
tions appears to be advantageous. 


It boosts responsiveness to the point. 


that all or virtually all persons 
can respond and results in no major 
sacrifice of validity. If anything, 


\ 

\ 
the use of pictures reduces the ten- 
dency to select the last option in 
either-or questidns. This appeared 
to be particularly true among severe- 
ly retarded persons, among whom re- 
sponse biases are generally more 
pronounced. Clearly either-or ques- 
tions ‘are not a panacea, for there 
are many topics that simply do not 
lend themselves to e either-or 
format. However, they should be given 
a great deal of consideration as 
viable alternatives tọ yes-no ques- 
tions. For example, ina severely 
retarded group one could count on 
getting a huge number "yes" an- | 
swers to the questions,\"Do you cook 
on a stove?" It would be possible 
to structure a pictorial ,either- 
or question on the same topic that. 
would in all likelihood --ield į 
formation that is much closer/to 
the truth; for example, "H 
a boy who cooks on the stove. Here 
is a boy who does not cook on the 
stove. Which boy is oe The 
one who cooks on stove or the 
one who does t cook on the stove? 
Point to the picture." 










i} 





In Chapter 9 we will spell out more 
of the implications of our findings 
for persons who solicit informa- 
tion from retarded persons by ques- 
tioning them. As the list presented 
here suggests, our study generally 
revealed more about how not to ask 
questions of retarded persons than 
how tc do so most effectively. Still, 
we believe that knowing what to 
avoid makes progress more iikely. 
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Chapter 9 


CONCLUSIONS AND IMPLICATIONS 


sin st Si he 
v MY \ 
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This report has covered h great 
deal of ground in detailing ‘the find- 
ings of’ a complex project that was 
designed to explore the feasibility 
of interviewing as a means of obtain- 
ing information directly from mental- 
ly retarded persons. In an attempt 
to bring closure to the report, we 
will briefly review the study and its 
major findings, discuss recommenda- 
tions for conducting survey research 
with mentally retarded persons, and 
point out some general implications 
for researchers and practitioners in 
the mental retardation field. 


3 


Summary of the Study 


WHAT DID WE DO? 

‘\We set out to answer a question 
originally posed by the president's 
Committee on Mental Retardation: Is 
a national polling of a sample of the 
mentally retarded population a fea- 
sible means of gathering consumer 
input? We approached this guiding 
question \by addressing the following 
more specific questions: 

À i 
aN 
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; 


1. To what extent can retarded 
Persons respond to questions 
in an appropriate fashion, and 
what factors affect their 
responsiveness? 


2. How reliable are their re- 
sponses, in the sense of being 
consistent over short periods 
of time? 


3. How valid are their responses, 
in the sense of being free of 
Systematic bias and agreeing 
with information provided by 
parents or caretakers or docu- 
mented in records? 


4. What types of questions appear 
to optimize responsiveness, 
reliability, and validity? 


We began with reviews of the litera- 
ture on the language and communica- 
tion skills nf retarded persons and 
response biases in survey research 
with the general population. We also 
explored issues in sampling and gain- 
ing access ‘to retarded persons for 
purposes of interviewing them. Our 
literature reviews suggested that, 


| 


| 
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for the most part, retarded persons 
are likely to behave linguistically 
like children at various stages in 
the development of mature language 
competence,--and that, moreover, many 
retarded persons have hearing and 
speech handicaps that may pose prob- 
lems in an interview. In addition, 
we found cause in the literature on 
response biases in the general popu- 
lation to expect retarded interview- 
ees, like children and less educated 
adults, to be especially prone to 
give biased answers; for example, to, 
acquiesce in response to yes-no 
questions. A 


Our xajor mission then bebame one of 
systematically testing int rviewing 
approaches with diverse samples of 
mentally retarded children and adults. 
Interview schedules focused on the 
topics of living circumstances, daily 
activities, utilization of services, 
and involvement in decision-making. 
Five samples of retarded persons were 
interviewed. The three primary sam-~ 
ples consisted of 52 institutional- 
ized children 12-16 years old, 58 
institutionalized adults, and 57 
children living in the community. 
Further research was done with 29 
institutionalized and primarily 
severely retarded children and adults 
whe nad proven largely but not total- 
ly unresponsive in the first inter- 
views, and ‘a small sample of 13 
adults living in the community and 
enrolled in a citizen advocacy pro- 
gram. 


Each study. also involved parallel 
interviews with "significant others"; 
parents, attendants, or citizen ad- 
vocates, depending on the sample. 
These persons were asked to answer 


questions paralleling most of the 
questions that had been asked of the 
retarded respondents so that the 
yesponses of retarded persons and 
nonretarded informants speaking for 
them could be compared. Finally, 
both institutionalized samples were 
interviewed twice, a week apart, so 


that the reliability of their re- 
sponses to the same questions asked 
en two differert occasions could be 
examined. In ali, well over 500 
interviews were conducted. Alterna- 
tive quéstions about the same topics 
were embedded within interview 
schedules to aid in identify‘ng 
question formats and phrasings that 
optimize responsiveness, reliability, 
and validity of response. Much of 
the analysis focused on responses to 
individual questions, comparing an- 
swers given at different times, an- 
swers given to alternative questions 


= on the same topics, and answers given 


‘by retarded persons and parallel 
answers given by parents or attend- ` 
ants. In addition, some composite 
scores were formed and used in 
correlational analyses so that in- 


~ dividual differences in interview. 


performance could be examined. 


To highlight some of the major find- 
ings, we will offer concise answers 
to the four major questions of the. 
study. 


TO WHAT EXTENT CAN RETARDED PERSONS 
RESPOND TO INTERVIEW QUESTIONS? 

l In general, we found that many 
retarded persons are capable of 
giving appropriate answers to simply 
worded questions. The measure of 
responsiveness to questions that we 
typically used was the percentage of 
occasions on which a response met, 
at least minimally, the-formal de- 
mands of the question (e.g., a "yes" 
or "no" toa Yes-gggquestion or 
mention of one activity in response 
to an open-ended question calling 
for an enumeration of activities). 
We obtained at least minimally appro- 


a priate responses at least two-thirds 


of the time in our samples of severe- 
‘ly to mildly retarded persons. Of 
‘the various inappropriate responses 
we encountered, the: most frequent 
appeared to be failure to give any 
resbonse; vague answers that could 
not be coded into content categories 
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(e.g., equivocation on yes-no’ ques- 
tions to the extent that the response 
could not be classified as "yes" or 


"no"), unintelligible responses, and _ 


irrelevant responses. An individ- 
ual's ability to respond appro- 
priately to questions was found to be 
highly stable from week to week, as 
irdicated by correlations of .96 
between percentages of answers that 
were apprcpriate in two interviews 
held a’ week apart with insvitutional- 
ized dhildren and adults. 


Most importantly, we found it pos- 
sible to predict now responsive 
individuals would be. First, respon- 
siveness appears to be a function of 
Ig. ' As we had expected, it is all 
but impossible to interview most 
profoundly retarded persons. In our 
one attempt to interview, profoundly 
retarded persons, only three of the 
16 adults approached passed a simple 
screening interview and supplied any 
answers at all to the full iftterviews. 
Severely retarded persons were di- 
verse in their ability to respond tc 
questions, and the extent of their 
limitations compared to moderategy 
and mildly retardec persons varied 
somewhat from sample to sample. 
Generally, however, responsiveness 
was greatest in the moderately and 
mildly retarded ranges. Overall, 

the correlations between IQ and re- 
sponsiveness scores in the three 
major samples varied from .35 to .67. 


Second, the ability to respond is a 
function of the type of question 
asked. Corsistently in our samples, 
high numbers of interviewees were 
able to respond appropriately to 
simple yes-no questions and to ques- 
tions involving choices among pic- 
tures in which a nonverbal response 
; (pointing) would suffice. Either-or 
‘questions were the next easiest to 
answer, and open-ended questions and 
multiple choice questions with three 
or four structured options were the 
most difficult for interviewees to 
answer. 


Finally, we found tnat responsive- 
ness could be increased by simply 
repeating a question if tne first 


_response was less than minimaliy 


appropriate. In fact, some persons 
became more responsive to questions 
after having been interviewed with a 
similar instrument a week before. 
All of this suggests that-fetarded 
persons may need additional training 
and experierce in beirg interviewed 
in order to learn hew to respond 
appropriately, but that. even during 
an interview they may benefit from 
additional chances to compreher.i the. 
question through having it repeated. 


However, one of the major assump- 
tions of the study was that obtain- 
ing answers is not enough. One must 
also determine whether answers given 
are reliable and valid. 


j 


HOW RELIABLE ARF ANSWERS? 

In general, we can say that the 
answer given one week by a mentally 
retarded person will typically 
correspond to the answer to the same 
question given a week later. In 
both institutionalized samples, an- 
swers given a week apart were gen- 
erally consistent cver 80% of the 
time. This was certainly true of 


` most yes-no, either-or, and open- 


ended questions. However, we found 
reliability very low for picture- 
choice questions about satisfaction 
with institutien life and for multi~ 
ple \choice questions inyoiving 
quantitative terms such as "a lot" 
and "sometimes." Finally, altnough 
we had expected answers to subjective 
questions to be relatively change- 
able, reliability figures for sub- 
jective yes-no questions were no 
lower than those for factual yes-no 
questions. 


Overall, then, one can generally ex- 
pect the picture of a group obtained 
one week to’ correspond to the picture 
of that group obtained the next week. 
One can expect retarded individuals to 
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_ individual differences. 


"stick to their guns" in answering most 
of the time. We were\not able to iden- 
ify individual, characteristics such as 
IQ that are closely related to the ex- 
tent to which an individual: responds 
reliably. , Moreover, we encountered 
many problems in interpreting reliabil- 
ity figures. And finally, as any re- 
searcher knows, reliability is a l 
necessary precondition of validity 

but is not sufficient to establish 
validity. Thus, we had to attempt to 
determine whether answers given were 
not only reliable but also valid and 
meaningful. 


HOW VALID ARE ANSWERS? 

The issue of response validity 
was addressed through two approaches. 
First, we compared responses of re- 
tarded persons to independent sources 
of validating information. Typically, 
this took the form of comparisons 
with the responses of attendants or 
parents to parallel questions, al- 
though some answers could also be 
compared to documented fact. Overall, 
retarded interviewees and attendants 
or parents agreed (gave matchiyg 
responses) about two-thirds of the 
time in our various samples,‘ At 
times, despite disagreement between 
individual pairs of clients and in- 
formants, pictures of the sample 
obtained from the two sources did not 
differ greatly. In such cases, one 
could at least use client and infor- 
mant data interchangeably to de- 
scribe a sample or to do analyses of 
At other 
times, as a consequence of disagree- 
ment between pairs of clients and 
informants, pictures of the group 
obtained from the two sources were 
widely discrepant. As the overall 





_finding of agreement about two- 


thirds of the time implies, it cannot ~: 
be safely assumed that interviews- 

with retarded persons will yield the 
same information as interviews with 
their caretakers or parents. This 

is true even when there is no reason 


to suspect that clients' responses 
are systematically biased. It is 
particularly true when there is 
reason to believe that response 
biases such aS acquiescence are at 
work. 


. Agreement with parents or attendants 


proved to be an imperfect index of 
response validity for two reasons. 
First, the two groups simply had 
different perspectives on some issues, 
and we were unwilling to infer that 
the retarded respondents were "wrong." 
Second, even high agreement was not 
always conclusive evidence for the 
validity of the responses of mentally 
retarded persons, for their answers 
sometimes revealed the workings of 
systematic bias. Thus, we took 
systematic biases in the response 
patterns of mentally retarded respond- 
ents into account in assesssing re- ` 
sponse validity. We did so princi- 
pally by comparing the responses of 
retarded persons to two or more 
questions on the same topic but with 
different wordings or formats. Where 
changing the way the questions were 


„asked powerfully altered the content 


of responses, we inferred that 
systematic biases were reducing 
validity. Systematic rather than 
random discrepancies between the 
responses of retarded respondents 
and parents or attendants were inter- 
preted in the same way. For example, 
we typically found clients far more 
likely to answer "yes" on yes~-no 
questions than informants were, and 
this pattern indicated that ac- 
quiescence on the part of retarded 
persons represented a threat to va- 
lidity of response. 


‘In. short, we combined agreement 
figures and evidence for systematic 
response biases in assessing response 
validity. In the case of yes~-no 
questions, in spite of relatively 
high agreément figures, clear evi- 
dence of acquiesc led us to con- 
clude that the responses clients 
are largely invalid. Even when 
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agreement was high, we typically 


‘found more clients saying "yes" than 


attendants or parents.. Moreover, 

in response to pairs of yes-no ques- 
tions with reversed wording (e.g., 
"Are you usually happy?" vs. "Are you 
usually sad?"), retarded respondents 
frequen‘ ly contradicted themselves 

by responding "yes" to both questions 
of the pair. Quite importantly, 
correlational analyses demonstrated 
that agreement percentages for ques- 
tions increased as the percentage of 
parents or attendants saying "yes" 
increased; clear indication that 
agreement figures were often artifi- 
cially high simply because acquies- 
cence on the part of clients happened 
to put them in agreement with infor- 
mants. 


Open-ended questions provide another 
example of a question format for 
which high agreement was not 
necessarily good evidence for re- 
sponse validity. The vast majority 
of agreements involved both client 
and informant failing to mention 
something in a category of response. 
Somewhat more encouraging results 
were obtained with simpler open- 


. ended questions that called for one 


answer rather tian an enumeration of 
responses. Multiple choice ques- 
tions with quantitative response 
alternatives (a lot, some, not much, 
etc.), as well as multiple choice 
questions calling for a selection of 
happy or sad faces to indicate a 
degree of satisfaction, appeared to 
have very Hittle validity at all. | 
Finally, either-or questions, as well 
as factual multiple choice questions 
with discrete rather than quantita- 
tive options, appeared to have rela- 
tively good validity. Despite some 
evidence of a tendency to prefer the 
last option in such questions, there 
typieally was not a marked bias in 


-the inconsistencies that occurred. 


Thus, we emerged from this analysis 
most confident about the validity of 
responses to open-ended questions l 


calling for a single answer, either- 
or questions, and multiple choice 
questions with discrete response 
alternatives. Ta 


We should note that analysis of re- 
sponses to questions whose answers 
were documented fact did little to 
increase our confidence in answers 
given by retarded persons. Discour- 
agingly high proportions of respon- 
dents were not able to supply fully 
correct answers to questions about 
their full names, birthdates, 
addresses,.and so on. Many others 
could not respond at all to such 
questions. Analysis of these ques- 
tions bolstered the conclusion that 
the validity of information provided 
by retarded persons can never be 
taken for granted. Moreover, it 
suggests the practical need for 
training retarded persons so that, 
particularly in emergencies, they 
can communicate basic facts about 
themselves accurately. 


Finally, we were generally unable to 
predict which persons would tend to 
give valid answers and which would 
not, Surprisingly, children living 
in the community were no more likely 
to agree with their parents (at least 
on a ‘set of yes-no questions) than 
institutionalized persons were to 
agree with their attendants. Persons 
with higher IQs were more likely to 
agree with their parents or attend- 
ants than persons with lower IQs, 
but the correlation was not strong 
enough that we could confidently 
predict whose answers can be trusted 
and whose cannot. Until further 
research is conducted, one must be 
wary of responses given by any re- 
tarded person, especially on the 
types of questions that are most 
likely to elicit systematic response 
biases. 


WHICH QUESTIONING APPROACHES ` 
ARE MOST USEFUL? 

A major undertaking in this proj- 
ect ‘was to conduct head-to-head 
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competitions between alternative 
question formats and phrasings in 
order to determine, with the topic of 
questioning controlled, which alter- 


native approach optimized responsive- 


ness and response validity. ; To 
examine response validity, we again 
compared answers given by retarded 
persons and either attendants or 
parents, and we also looked for signs 
of systematic bias when the answers 
of clients to alternative questions 
were compared. , Since Chapter 8 
included a rather detailed summary 

of the findings of these analyses, we 
will only briefly review the. recom- 
mendations that. were made. 


l. Careful attention should be 
devoted to question wording. 
Seemingly small changes in 
wording can have large effects 
on responses. For instance, 
the use of examples of the - 
topic of inquiry in the ques- 
tion (e.g., mention of football 
and baseball as examples of 
sports to be enumerated) 
biases responses such that the 
examples are mentioned more 
often than they would other- 
wise be. 


2. In probing for additional in- 
formation on open-ended ques- 

` tions by asking "What else?" 
the probing should not be 
carried on indefinitely until 
the respondent can think of no 
more, for this strategy appears 
to threaten validity of re- 
sponse. It is probably wiser 
to use such probes only once 

€ or twice. ` Moreover, when 
probes are used, they should 
restate the content of the 
original question to increase 
understanding (e.g., "What 
else do you usually do for 
fun?" rather than "What else?"). 


3. Multiple choice questions with 
quantitative response options 
which are designed to get at 


extents of involvement in 
various activities appear to be 
generally useless, for retarded 
persons appear to have diffi- 
culty attaching consistent 
meanings to such terms as "a 
lot," and "not much." Simi- 
larly, pictures of happy and 
sad faces do not aid in elicit- 
ing information about satis- 
faction with life. However, 
multiple choice questions about 
basic facts that offer discrete 
response alternatives (e.g., 
walking, riding the bus, riding 
a bike, and riding in a car as 
ways of getting to school) do 
appear to be useful. 


Yes-no questions should prob- 
ably be avoided in interview 
ing retarded persons because 

of the often disastrous effects 
of acquiescence on response 
validity. Pictures as accom- 
paniments to such questions 


apparently do nothing to. 
prevent acquiescence. 


Open-ended questions have the 
major disadvantage of not being 
answerable by high proportions 
of mentally retarded persons. 
Qur tests of questions calling 
for enumerations of activities 
suggested that these questions 
yield very little information 
from most people and probably 
result in underreporting of the 
activities inquired about. By 
contrast, open-ended questions 
calling for single answers l 
generally yield valid responses 
assuming they can be answered, 
and appear to\be appropriate in 
interviewing higher IQ retarded 
persons, 







Either-or questions emerged as 
the single most| satisfactory 
type of questioning strategy. 
Although they are not quite as 
easy to answer as yes-no quus- 
tions (indeed, /several respon- 
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dents answered them with a 
"yes"), responsiveness to them 
was still relatively good, and 
more importantly, clients' 
answers and informants' answers 
tended to be in agreement.’ The 
only negative finding wasa 
tendency, but not a strong or 
totally consistent one, toward 
selection of the’ last of the 
two options presented, a re- 
sponse bias that would need,to 
be guarded against. Finally, 
it was in conjunction with 
either-or questions that pic- 
tures appeared to have their 
greatest effect. The use of 
pictures with either-or ques- 
tions consistently increased 
responsiveness to the questions 
‘without sacrificing validity 
(and in a few cases enhancing 
it by counteracting the bias 
in favor of the last response 
option). 


we are painfully aware of the fact 
that our research has typically told 
us a great deal more about how not 

to interview retarded persons than 
how to do so effectively. As the 

list above suggests, we continually 
encountered threats to response 
validity that forced us to raise 

grave questions about the utility of 
some questioning tactics. Even for 
the best of question formats, validity 
was often marginal. 
findings do offer guidelines for con- 
structing interview schedules ,to 
optimize the chances of getting an- 
swers, and more importantly, getting 
answers that are meaningful reflec- 
tions of the attitudes, needs, and 
circumstances of retarded citizens. 


DIRECTIONS FOR FUTURE RESEARCH 

The present study breaks new 
ground by exploring in a systematic 
way methodological issues in inter- 
viewing retarded persons. Obviously 
the answers we have been able to 


provide are less than complete; more- 


Nonetheless, the ` 


i 


over, our study has raised at least 
as Many questions as it has answered. 
It has at least established unam- 
biguously the fact that the validity 
of answers given by retarded persons 


_can never be taken for granted. 


However, many of the specific factors 
which influence the responses of 
mentally retarded persons remain un- 
explored. We would encourage other 
researchers in the mental retardation 
field to continue this promising line 
of research in hopes of enabling both 
researchers and practitioners to 


` obtain useful information from re- 


tarded persons. We recommend careful 
investigation of the entire range of 
factors that may affect. responses. 


Our study has focused almost exclu- 
sively on characteristics of inter- 
views that affect responses, primari- 
ly question format and phrasing. 
Other issues in, interview design 
remain to be addressed. For example, 
our study suggested that social , 
desirability may enter into the 
responses of retarded persons to many 
questions, but we did not systemati- 
cally deal with that issue.. Do 
mentally retarded persons have a 
heightened motivation to give social- 
ly approved answers, and are their 
responses especially biased on items 
that raise issues of self-presenta- 
tion? A careful investigation of 
question difficulty, perhaps empha- 
sizing vocabulary difficulty and 
conceptual complexity, might also 5 
prove fruitful, as this was only in- 
cidentally examined in;the present 
study. ~ d 


Moreover, as Sudman and Bradburn 
(1974) indicate, respondent variables 
and interviewer variables, along with 
task variables such as question 
design, can eee error into 
survey research. We have explored 
the respondent variable of IQ in 

some detail, and have given some 
attention to other’ respondent vari- 
ables such as age, sex, and institu- 
tionalization. Although we found 
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these additional respondent variables 


to be of minor importance, more 
intensive investigation of them might 
be warranted. The fact that we have 
found lower IQ individuals less 
responsive, more likely to acquiesce, 
and somewhat less likely to agree 
with parents and attendants empha- 
sizes the potential importance of 
respondent variables. As for inter- 
viewer variables, we left the issue 
entirely unexplored. For example, 

do interviewers who are already 

f miliar to retarded interviewees 

oD ain different information than do 
outsiders? How do the interviewer's 
attitudes and behaviors affect 
responsiveness and the nature of 
responses? The literature on threats 
to response validity in survey re- 
search with the general population 
(See Chapter 2) provides a ready 
source of leads. 


Meanwhile, researchers who are not 

directly interested in methodological 
issues in interviewing cannot afford 
to ignore them if they are conducting 


_ research with retarded persons that 


entails asking questions. Again, 
the literature on survey research, 
as well as the present monograph, 
can provide guidelines to follow in 
constructing instruments. The re- 
searcher sensitive to the issues 
will probably want to build checks 
on response validity into whatever 
instruments are developed. We are 
confident that researchers will 
detect many other leads in this- 
monograph that we have not mentioned 


here. 


How to Interview 
Mentally Retarded Persons 


Based on our experience as well 
as our data, we would make the fol- 
lowing recommendations to persons 
seeking information from mentally 
retarded individuals through survey 
research. (The reader is also 


referred back to Chapter 4 for more 


about the logistics of interviewing.) 
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PREPARATION 

Invest a substantial amount of time 
in preparation before any "real" 
interviews are conducted. 

It is not unusual in the mental 
retardation field or in any other 
area of research for researchers to 
"slap together" an interview or 
questionnaire. There is not a 
strong tradition in survey research, 
as there is in other areas of 
measurement such as testing, of 
establishing the reliability and 
validity of instruments. We have 
"slapped together" interview sched- 
ules for mentally retarded persons 
ourselves. While we pilot tested 
many questions used in the present 
study, our efforts could still be 


‘.. characterized as informal at best. 


Granted, we labored individually and 
in work groups for hours deciding 
what we wanted to ask and how to 
phrase and structure questions on 
those topics. We went through many 
draft versions of these interviews 
before we were satisfied. But, most 
of our planning was based on armchair 
thinking, on an attempt to imagine 
all the problems a question might 
present to a mentally retarded per- 
son. There were still numerous 
occasions in which we found, while 
administering interviews, that we 
had not anticipated everything, that 
we were getting bad information be- 
cause our question was bad. To give 


just one example, we found that 


institutionalized persons often had 
difficulty with questions about their 
activities due to the fact that they 
were not sure whether we meant to 
ask them about what they did at the 
institution, what they did at home, 
or both. A few interviewees ex- 
plicitly asked which we meant, but 
moSt others simply answered, and 
there is no way to tell how they 
interpreted the questions. If we 
had wanted to know their extent of 
involvement in, for example, home- 
making activities in the cottage at 
the institution, we would have had- 
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to add the phrase "in your cottage" 
or the like to each question. We 
indeed did just that once we recog- 
nized that our initial questions were 
ambiguous, but by then we had col- 
lected data from whole samples and 
had no way to be sure whether their 
answers pertained to activities at 
the institution, at home, or both. 


We would recommend starting with a 
serious "armchair" process of de- 
signing what appear to be simple 


~~ unambiguous questions about 


whatever topics are of interest. 
However, we would then recommend 
moving from the armchair to the 

ield to do extensive piloting of 

he draft interview schedules with 
réal retarded persons similar to 
those to be interviewed in the full- 
fledged study. No matter how well 
one anticipate exceptions, misinter- 
pretations, and ambiguities, one is 
certain to detect more during the 
piloting process. If it is feasible, 
we would recommend conducting 
parallel interviews with attendants, 
parents or houseparents, or obtaining 
validating data from files and 
records so that the pilot testing 
process yields information about the 
validity of answers, beyond what can 
be obtained by seeing how inter- 
viewees are interpreting questions. 
Because interviewing retarded persons 
is particularly difficult, this kind 
of preparatory work appears to be 
crucial. 


QUESTION DESIGN 
Draw on the successes and failures 
of the present study in designing 


‘ questions. 


To be as brief as possible here, 
that would include paying careful ~ 
attention to question wording and 
using question formats that are 
likely to optimize responsiveness 
and validity of response (see Chapter 
8 for more details). We would rec- 
ommend an interview schedule that 
is highly structured in basic design, 


‘question. 


with open-ended questions used as 
follow-up questions in areas where 
more clarification is desired from 
more verbal respondents. We would 
recommend heavy use of either-or 
questions and multiple choice ques- 
tions with discrete rather than 
quantitative options. If severely 
retarded persons are to be interviewed, 
such questions should, when possible, 
be accompanied by pictures. Simple 
open-ended questions calling for one 
piece of information rather than an 


-enumeration would also be appropriate, 


although it must be recognized that 
many less/ verbal persons will not be 
able to answer them, even if they 
concern very basic facts. By using 
structured questions as the basis for 
the interview schedule, one can hope 
to obtain at least some information 
frou virtually all persons in the 
sample. This information can be 
supplemented with elaborative and 
illustrative material gained from 
open-ended questions that only some 
respondents can answer. We would 
also recommend asking as few ques- 
tions as possible that require con- 
cepts of number, time, quantity, 
monetary value, history, and the like. 


Based on piloting, one can anticipate 
problems that interviewees are likely 
to have and design, in advance, 
paraphrases or alternative phrasing 
of questions to be used if inter- 
viewees either cannot understand or 
ask for clarification of the original 
Alternatively, the deci- 
sion can be made to ask questions 
only in one standard phrasing and 
simply live with any missing data 
that result. The advantage of the 
first a.ternative is that one can 
hope to gather information from a 
larger portion of the sample under 
study; the risk is that the alterna- 
tive way of asking for the informa- 
tion may change responses. The nore 
like the original question the 


-clarifying question is, the less that 


risk will be. Indeed, our analysis 
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of responsiveness suggests that 
simply repeating the question in the 
same form will increase responsive- 
ness, with no altération of the 
meaning of the question. ‘Thus, such 
repetitions are clearly recommended. 
Still, there are inevitably some who 
cannot understand the question even 
when it is repeated. 


Similarly, probes and follow-up 
questions generally should be 
planned in advance. A sensitive 
interviewer can do wonders in pur- 
suing an answer and drawing a person 
out; as a reilt, we have a reluc- 
tance ‘to ad ze total standardiza- 
tion. However, our concern with the 
potential impacts of question format 
and phrasing on responses makes it 
necessary. Our recommendation that 
interview schedules be layered, with 
structured questions followed up by 
open-ended ones, helps in this re- 
gard. The structured questions would 
provide exactly the same stimuli to 
all respondents. Then to gain 
further detail from more verbal 
subjects, the interviewer might be 
given discretion to pursue interest- 
ing responses, as long as that inter- 
viewer is sensitive to the effects 
of question format and wording and 
will not lead the respondent. 


We have recommended that ves-no 
questions be avoided if at all pos- 
sible, and will stick with that rec- 
ommendation. However, if they are 
used, we would strongly urge that 
checks for the operation of acquies- 
cence be built into the interviewing 
process. Our data on acquiescence 
have suggested that it is not just a 
matter of isolating a few acquies- 
cers; we have found instead that 
most retarded persons will acquiesce 
to some extent at some time. Still, 


-the phenomenon is more pronounced 


for some persons. One possible check 


‘is to scatter oppositely worded 


questio.s of the type we have used 
throughout the interview schedule 
and determine how many times respon- 


, sons in lower IQ ranges. ; 
\ attempt to interview'profoundly re- 


dents contradict themselves by 
answering "yes" to both questions in 
a pair. The other strategy we used, 
asking questions that should properly 
be arswered "no," appears to be 
equally appropriate. One might wish 
to develop a formal acquiescence 
scale using one or both of these 
approaches, testing it out in advance 
to make sure it is reliable and valid 
Even-an informal check, however, 
would permit detection of frequent 
acquiescers. We are not yet ina 
position to suggest an explicit. 
criterion for defining, frequent 
acquiescence. It would be up to the 
researchers involved to ‘establish. - 
such a criterion and then\use it ‘to 
eliminate acquiescers from the usable 
sample. However, because checks for 
acquiescence would inevitably mean 
disqualifying some responses, we 
continue to think that ‘it is best to 
avoid asking yes-no questions if at 
all possible. 


HEARING FROM SEVERELY AND PROFOUNDLY 
RETARDED PERSONS | 

Use alternative ways of obtaining 
information about and from more 
severely retarded persons. 

While we have not always found 
strong relationships between IQ and 
-the. interview measures we have 
examined here, we have generally 
found more problems interviewing per- 
Our one 


tarded persons, done simply to docu- 
ment the obvious, was an almost 

otal failure. Severely retarded 
ersons, while less predictable, also 
have relative difficulty answering 
verbal questions. For them, w found 
the pictorial either-or format „arti- 
cularly promising, but we must still 
conclude that they have limited 
capacities to respond and that they 
are particularly susceptible to re- 
sponse biases such as acquiescence 
and a preference for the last option 


- in an either-or question. The 
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two-layered interview schedule 
‘approach we have advocated here 
would perhaps allow one to obtain 
answers to structured questions from 
most severely retarded persons, but 
few of them would be able to elabo- 
rate on their answers in response to 
open-ended follow-up questions. 
Thus, there appears to be no way to 
avoid underrepresenting the per- 
spectives of more severely retarded 
persons in survey research. 


There are alternatives. One which 
has been used before is to obtain 
the desired information from a 
parent, attendant, houseparent, or 
other respresentative of the retard- 
ed person. However, there are 
limitations to this approach. We 
have at times encountered cases in 
which attendants or parents and 
retarded respondents simply disagreed 
about something, where one cannot 
explain discrepancy between their 
answers aS a matter of response bias 
on the'part of the retarded respon- 
dents. We have also found numerous 
instances in which attendants and 
parents have surprised us by not 
knowing something that we assumed 
they would know about the retarded 
persons they interact with. 


On the other hand, we feel that 
relying on,such persons as informants 
is better than simply writing off 
severely and profoundly retarded 
persons. Indeed, even when mildly 
and moderately retarded persons are 
interviewed, we would recommend 
interviewing their "significant 
others" as well, partly as a check 

on the validity of certain responses, 
and partly to obtain information that 


most retarded persons have difficulty ` 


providing themselves (e.g., informa- 
tion about history, involvement with 
service agencies, financial matters, 
and amounts of time involved in 
various activities). Parents and 
care providers are especially valu- 
able as sources of factual informa- 
tion; whether they can provide 


information about retarded persons' 
perspectives and opinions is another 
matter. We would urge that severely 
retarded persons still be given an 
opportunity to speak for themselves, 
particularly about their opinions, 
even when nonretarded informants 
must be the primary source of in- 
formation about them. 


Another alternative to interviewing 
low verbal individuals is behavioral 
observation. Instead of asking them 
about daily activities, one might 
observe them through :time sampling 

in the residential setting.: Instead 
of asking about satisfaction with the 
residential setting or with service. 
providers,. one could potentially 
observe affective responses to ser- 
vice providers or to events in the 
residential setting. We are not 
claiming that this would be easy, for 
behavioral observation is time-con- 
suming and difficult. However, that 
might be the best way, coupled with 
interviews of informants knowledge- 
able about the retarded person, to 
approximate an understanding of the 
needs, perspectives, and circum- 
stances of persons with limited or non- 
existent verbal skills. 


To be sure, the data would not be 
comparable to data obtained from more 
verbal individuals through interviews. 
Thus, we have no easy solution to the 
problem of obtaining comparable data 
from the entire range of retarded 
persons. When previous researchers 
(e.g., Bell, 1976) have relied on 
informants to answer for low-IQ 
persons but have’ obtained answers 
directly from higher-IQ persons, 
there is simply no way to tell 
whether any resulting differences 
between high and low-IQ persons are 
due to real differences in their 
lives or due to the difference 
in data sources. This same quandary 
would be true any time the source 

of information for one group is 
different from the source of infor- ~ 
mation for another. However, it 
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would be possible to interview both 
clients and nonretarded informants 
in the high IQ range, and use the 
agreement berween their responses as 
a way of roughly estimating the 
degree to which responses given by 
nonretarded informants and severely 
retarded persons might agree. If 
agreement is high, one would feel 
more comfortable combining nonre- 
tarded informants' data for low-IQ 
persons with retarded informants' 
data for high-IQ ones. 


We should perhaps make some mention 
of the nonverbal communication 
systems now used with language- 
deficient retarded persons. In our 
opinion, while such systems are per- 
mitting great advances in the com- 
munication training of severely and 
profoundly retarded persons, the 
time has not yet arrived when they 
can be used to collect daa from 
samples of retarded persons on a 
widespread basis. The major problem 
is that even if interviewers could ' 
be trained to administer the inter- 
view in a nonverbal communication 
system, what: nonverbal communication 
system would be used? There is no 
standardization at this point, as 
.there would be if one wanted to 
interview deaf persons and could use 
Ameslan as the vehicle. While 
Ameslan has been used with some 
retarded persons, many »əther systems, 
including sign languages developed 
especially for use within a particu- 
lar facility, are too provincial’ to 
be used in a large-scale interviewing 
project. It might be possible to 
enlist the help of facility staff to 
"translate" interview questions inco 
a local nonverbal communication 
system. However, one would then 
have to guard against the possibility 
that respondents would react dif- 
ferently to a staff interviewer 

than to an cutside interviewer. All 
things cons: Jered, then, these 
systems do ot offer much hope at 
the present time, except, of course, 


as a means for facility staff to 
gather input from their own residents 


for program planning purposes. 


E Implications of the Study 
for the Mental Retardation Field 


THE FEASIBILITY OF A NATIONAL POLLING 
This project was, inspired by the 
President's Committee on Mental 
Retardation, which wanted to know 
whether or not a periodic national 
polling of retarded citizens would 
be feasible as a means of obtaining 
consumer input. In light of our 
findings, the answer to this question 
depends on the form this proposed 
polling takes. A comprehensive sur- 
vey of mentally retarded consumers, 
comprehensive in the sense of involv- 
ing the whole range of mentally 
retarded persons, and comprehensive 
in the sense of covering a broad 
spectrum of topics relevant to the 
lives of the retarded, appears not 
to be feasible. Very sin.°, too 


-~ Many retarded persons woui? ave too 


much difficulty providing valid in- 
formation about too many topics. 
However, less inclusive surveys, 
cortstructed and administered with our 
findings in mind, could provide PCMR 
and others with valuable information 
about the needs, circumstances, and 
attitudes of particular groups of 
retarded consumers. 


The data suggest that a standardized, 
comprehensive, national, survey of 
mentally retarded persons is imprac- 
tical on two major grounds. First, 
it is clear that standardized inter- 
viewing of the entire spectrum ‘of 
mentally retarded consumers is not 
possible. Our experience indicates 
that interviewing profoundly retarded 
persons is clearly impracticable, 
and that the population of severely 
retarded consumers includes many 
people who are unresponsive or pro- 
vide responses of very low validity. 
Further, even among the mildly and 
moderately retarded, many persons 
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give responses to interview ques- 
tions which are largely invalid. 
Serious attempts to interview re- 
tarded ‘persons must include validity 
checks which screen out the answers 
of people who consistently display 
response biases. Thus, survey re- 
search with retarded consumers 
necessarily. involves a selective 
sample of more verbally proficient 
and intelligent persons, and the 
information gained might be quite 
unrepresentative of the entire 
national population. 


Secondly, the data presented in the 
preceding chapters indicate that 
there are serious restrictions on the 
types of information which can be 
obtained from interviewers with men- 
tally retarded respondents. The only 
question formats which generated 
information of reasonable validity 

in our project were either-or ques- 
tions, multiple choice questions 
offering discrete rather than quanti- 
tative response options, and open- 
ended questions targeted at discrete 
bits of factual information. Re- 
striction of an interview schedule 

to these formats limits the types of 
information that can be sought. 
Further, we have found that certain 
types of content areas are especially 
problematic; specifically, questions 
using such concepts as time, number, 
quantity, monetary value, history, 
and so on are difficult for mentally. 
retarded persons and tend not to 
generate meaningful answers. Thus, 
we cannot conduct a polling of re- 
tarded consumers wh.ch is comprehen- 
.Sive with regard to topics included. 


Our study indicated unambiguously that 
retarded persons would have too much 
difficulty providing valid informa- 
tion for a comprehensive national 
survey to be feasible. We have not 
discussed some of the formidable 
problems in constituting a national 
sample; distinguishing between the 
identified population and the "actual" 
` population would be difficult, and 


gaining access to even the identified 
population, given current safeguards 
protecting the confidentiality of 
client records, would be virtually 
impossible. 


However, even if a comprehensive 
national survey appears to be a poor 
means of information gathering, the 
goal of hearing from retarded individ- 
uals remains an important and viable 
one. We would be enormously discour- 
aged if our study caused people who 
work with the retarde to give up; to 
stop trying to hear from their 

clients regarding crucial issues in 
their lives. Rather, implicit in our 
statement of what we cannot do is a 
host of information regarding what we 
can do, and it is to the possibili- 
ties for communicating with mentally 
retarded consumers that we turn next. 


THE NEED FOR PERSISTENCE 

Even though we have had, at times, 
to be pessimistic about interviewing 
retarded .persons,. our data do indicate 
that one can interview some of the 
people some of the time. It is our: 
hope that this monograph, rather than 
discouraging people from attempting to 
gather input directly from mentally | 
retarded cdnsumers, will encourage them 
to do so in a methodologically sound 
way: As we see it, the most serious 
problems we have encountered are those 
that involve systematic response biases. 
Acquiescence is the single most 
serious problem we have encountered, 
and efforts to guard against it must 
certainly ,be made. However, when 
there is no systematic response 
effect operating, the threat to 
validity is usually not as great. 
Granted, there still may be error in 
the data of substantial size, but 
the picture one obtains of a group 
through one questioning technique 
may not differ greatly from the 
picture one obtains from an alterna- 
tive technique. At other times, we 
have found retarded persons able to 
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answer and able to answer with valid 
information. Furthermore, we have 
suggested some ways of optimizing 
responsiveness and response validity 
which we hope will be useful to 
others. We have assumed and continue 
to assume that because communication 
is a two-way street, communication 
with retarded persons can be improved 
if interviewers improve their tech- 
niques of soliciting information. 


However, our experience with retarded 
persons has instilled in us a com- 
mitment more basic than our commit- 
ment to increased methodological 
sophistication. That commitment is 
to an increased sense of humanity 
and sensitivity in our dealings with 
mentally retarded persons. Our 
empirical data somehow miss a vital 
point about our interactions with 
the retarded: namely, that nearly 
every one of our respondents, even 
low verbal individuals, had some- 
thing to say. Repeatedly, people 
we interviewed told or showed us 
that being interviewed was for them 
a rare, exciting, and gratifying 
experience. Perhapsthis is a commen- 
tary on the fact that their views 
are often ignored, or at least that 
nonretarded people rarely sit down 
for any length of time with them 
individually and ask them to talk 
about themselves. For the inter- 
viewer who enters into the task with 
a genuine regard for retarded people 
..and'a real desire to learn about 
them, the interviewing process is 
equally rewarding. In fact, we 
developed a strong sense’ that in 
those interviews which did not elicit 
much information, the failure result- 
ed not because the mentally retarded 
interviewees did not have the right 
answers, but because we did not ask 
the right questions. Our structured 
interview schedules not only taxed 
the communication skills of our 
interviewees, but often failed to 
give them a genuine opportunity to 
speak their minds. A more flexible 


‘amenable to formal, 
interviews, we recommend careful 


approach, tailored to the capabili- 
ties and interests of the individual 
client, frequently produces a less 
rigorous but richer interaction. At 
least we found this to be the case 
in our more free-wheeling interviews 
with adults living in the community. 


Thus, in its entirety, our study 


provides a set of more or less specif- 
ic recommendations for the field of 
mental retardation. First, the idea 
of a comprehensive national polling 
of retarded consumers is probably not 
practical, at least given our current 
knowledge. However, a segment of the 
retarded population can respond to 
traditional, standardized interview 
techniques, and can give valid in- 
formation if interviewed within the 
constraints discussed in the previous 
section. We suggest that workers in 
the field intensify efforts =o inte: - 
view, with methodological rigor, 
particular groups of mentally retard- 
ed consumers of services. In addi- 
tion, even when formal interviewing 
is not possible;.given the person or 
the topic, retarded persons may still 
have a great deal to say. For those 
individuals and’ topic areas not 
standardized 


efforts to communicate in less struc- 
tured, less formal interviews, 
tailored to the needs and capabili- 
ties of the individual client. Our 
project generated some useful guide- 
lines as to how this process of in- 
formal communication can be made 
more efficient and can generate more 
valid information. It is to the 
guidelines that we can turn next. 


ISSUES FOR CONSIDERATION 
BY PRACTITIONERS 

For the most part, this mono- =; 
graph has been directed toward re- 
searchers and others who conduct 
formal interviews with retarded per- 
sons for data-gathering purposes. 
Yet there is no great difference, 
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except for degree of formality, 
between a survey research project 
and the kinds of informal question- 
ing and interviewing that case - 
workers, counselors, and other pro- 
fessionals do daily with retarded 
clients. All major legislation 
affecting retarded persons today, 
the Education for All Handicapped 
Children Act and the 1978 rehabili- 
tation amendments, including the 
develcpmental disabilities sections, 
calls for the development of indivi- 
dual program plans. Moreover, they 
call for the involvement of clients 


or students, or their representatives, 


in the development of. such plans. 
Beyond the individual program plan- 
ning process, professionals who work 
with the retarded frequently 

need to obtain information from 
‘them and to interact with them 
verbally in order to carry out treat- 
ment and training efforts and to 
evaluate individual progress and 
overall programs. 


We view our findings as just as 
relevant to the practitioner as to 
the researcher. Take, for example, 
the matter of yes-no questions. 
Everyone who talks to a young child 
or a retarded person with limited 
verbal skills slips into using them. 
Yet our findings clearly indicate 
that retarded persons are highly 
likely to say "yes" to such ques- / 
tions, whether "yes" is the appro-/ 
priate answer or not, and that the! 
answers one obtains to two yes-no/ 
questions that are oppositely worded 
are quite likely to be discrepant as 
a result. We think practitioners 
should be alert to the possibility 
of acquiescence in their discussions 
with retarded clients, particularly 
if important decisions are going to 
_ be based on whether the client says 
"yes" or,"no." One might want to 
consciously attempt to re-ask a 
yes-no quéstion later in the inter- 
view with ‘the wording teversed (for 
example, after asking, "Do you think 


.the client really does want. 


the car wash would be a good place 
to work?" ask, "Do you think the car 


- wash would be a bad place to work?"). 


If the answers: are both "yes," one 

at least knows that further question- 
ing will be necessary to clarify what 
Alter- 

natively,.as we have suggested here, 

an either-or question ("Do you think 

the car wash is a good place to work 

or a bad place to work?") might be a 


.better way of obtaining a valid an- 


swer. 


The practitioner might benefit in 
other ways from being more sensitive 
to the effects of question wording 
and format. Use of the simplest 
wordings possible is a starting 
point. Again, it is useful to ask 
the same question in two or three ‘ 
ways to check on client comprehension. 
Wording is particularly problematic 
when one is asking about service 
agencies. Our studies, for example, 
have suggested that clients have 
varied understanding of wnat vocation- 
al rehabilitation is. If asked 
whether they have been a vocational 
rehabilitation client, retarded per- 
sons may not understand that wording, 
but may recognize "voc rehab," or 
"TRC," or another local name for the 
rehabilitation agency. (We have some 


evidence to suggest that they may 
be even more likely to talk about 
their experiences with an agency if | 
you are able to name the counselor 
they actually had. Many of our 
respondents related to a service 
agency in terms of a specific staff 
member, sometimes not being able to 
identify the agency or what its 
functions as a whole was called or was 
all about.) 


As we have also suggested, using 
examples to make a question more 
concrete: may backfire by changing 
the meaning of the question or by 
prompting mention of the examples in 
answers. Generally, then, one would 
want to be careful not to lead the 
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client toward a response that he or 
she might otherwise not give! 


When the same information is being 
sought from several clients (for 
example, in a program evaluation 
survey), the same standardization of 
questions that we would recommend 
for researchers should be used. One 


-often cannot predict how much a 


slight change in wording might, affect 
the answers that are given. On the 
other hand, we are quite confident 
that if some clients are asked a 
yes-no question and others are asked 
an either-or question on the same 
topic, their answers are likely to 
differ. This clearly points to the: 
need for careful planning in design- 
ing the questions to be used and ad-. 
herence to those questions in actual- 
ly conducting the interviews. The 
interviewer might still be given 
latitude to pursue interesting or 
confusing answers to the standard 
questions. To be sure, the prac- 
titioner does more informal types of 
data-gathering than the researcher, 
and does not have as much concern 
with methodological issues. But 

this does not mean that the practi- 
tioner does not need to be concerned 
with the validity of the answers 
that are given. 


% 


Finally, a note, to teachers\| and 
trainers of the mentally retarded is 
in order. Our findings are highly 
relevant to teaching communication 
skills to retarded persons. Overall, 
they suggest that many retarded 
persons are in need of training which 
would better equip them to understand 
and answer questions, not only in 
interview situations but in everyday 
exchanges with others. Too often, 

we feel, language and communication 
training emphasizes the formal rules 
of language but does not go far: 
enough to help retarded Students use 
language appropriately in different 
social contexts. Retarded eee 
need explicit training in answeying 





they are uninformative. 


questions appropriately so that their 
answers meet the demands of the 
questions (e.g., so that they answer 
an either-or question with one of the 
two options rather than by saying 
"yes™ as many of our subjects did). 
Beyond that, as our findings clearly 
suggest, they may neei guidance in 
answering questions accurately, es- 
pecially questions about basic per- 
sonal information. 


More specifically, we would offer the 
following suggestions for inclusion 
in communication training programs. 
First, concentrate on teaching re- 
tarded persons to answer questions 
appropriately and to avoid answers 
that are unintelligible, irrelevant 
to the question, or so vague that 

Our data on 
responsiveness to questions suggests 
a developmental hierarchy of types 

of questions, progressing from yes-no 


‘questions and questions involving 


pictures that can be answered by 
pointing, to either-or questions, to 
multiple choice questions, and final- 
ly to open-ended questions. Although. 
we ‘have not emphasized it here, there 
are\ also levels of difficulty within 
the category of open-ended questions. 
For example, in previous research | 
(Sigelman & Werder, 1975), retarded 
adults were more responsive to`’ 
what-questions asking them to. name 

or describe something than they were 
to why-questions and questions that 
asked them to consider hypotheti- 

cal situations (e.g., "What would 
make things better for you where you 
live?"). \ Training might involve 
asking,students progressively more 
difficult types of questions while 
giving them feedback and instruction 
as to what the question is demanding. 
Once the student can supply minimally 
appropriate responses, the trainer 
might encourage them to provide more 
complete responses and elaborate on 
their answers.: For instance, if the _ 
question is "Do you have a job?" a 
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minimal response might be "yes," a 
better response might be "Yes, I have 
a job," and an even better one might 
be "Yes, I work part-time at a cafe- 
teria.” 


In addition, training should be 
directed toward obtaining valid 
answers, not just answers. Two 
points are particularly worth empha- 
sizing. First, as we have shown, 
many retarded persons are not’ able to 


“—~-~answer basic questions about them- 


selves such as full name, birthdate, 
address, and sO on. For some, the 
problem’ is answering an open-ended 
question at all, due to limited ver- 
bal skills. However, we feel that 
even those with limited verbal skills 
could be taught to respond by rote 
to a set of basic questions about 
themselves. For others, the diffi- 
culty lies not in answering but in 
answering accurately, and these in- 
dividuals also need training. We 
feel strongly that retarded persons 
should be able (for example, in an 
emergency) to tell people such 
things as who they are, where they 
live, their age or birthday, their 
telephone number, and so on. 


Secondly, we feel that acquiescence 
and possibly other response biases 
should be given attention in com- 
munication training programs. One 
simple way to do this is to have 
students practice answering questions 
about concrete objects and materials 
or about verifiable facts in their 
immediate environments (e.g., "Is’ 
your shirt blue?" "Am I your fa- 
ther?"). This allows the instructor 
to provide feedback about the accura- 
cy of answers that are given as well. 
as their form. Quite obviously, re- 
tarded persons can get into trouble 


‘in the "real world" if they continual- 


ly say "yes" when they are asked 
questions calling for a "yes" or "no" 


answer. Rosen, Clark and Kivit: (1977) 


‘-have-exploréd this acquiescence prob- 


lem in the larger context of compli- 
ance with requests and have found, 


for example, that many retarded per- 
sons will agree to take a pill not 
knowing anything about what the pill 
contains. In every area of their 
existence, from sex to individual 
program planning, retarded persons 
must learn to avoid exploitation and 
assert themselves. At least part of 
this Larger mission can be accom- 
plished by teaching them not to say 
"yes" without thinking about the 
answer. 


CONCLUDING REMARKS 

We would like to leave our readers 
with a final message: retarded people 
must finally be treated as people. 
For service providers and researchers 
alike, this concern with basic rights 
includes an obligation to obtain in- 
formation concerning the perspectives 
of the retarded. Too often we have 
not bothere¢ to ask retarded people 
themselves what is going on and what 
they think of it. We sometimes 
assume too readily that a residential 


program or an educational activity 
or a service system that looks good 
to us must look good as well to the 
retarded clients-it serves. We have 
relied too often on the opinions of 
“significant others" to determine 
what retarded individuals need or 
want. Although data obtained direct- 
ly from retarded persons through 
interviews cannot totally replace 
other sources of information, it 
should be one of the ways in which we 
attempt to "know" retarded persons. 


,If questions are hastily designed 


without attention to measurement 
issues of the kind discussed here, 
the gains will be few. Indeed, the 
data collected might easily be mis- 
leading; for example, if program 
evaluators iask residents of a fa- 
cility questions like "Are you satis- 
fied with the training here?" they 
thereby assure themselves of a favor- 
able evaluation report. However, if 
care is taken in deciding how to 

seek information, the gains can be 
great. Furthermore, our data clarify 
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an important point: even informal 
interactions with mentally retarded 
persons. must be conducted with sensi- 
tivity to the level of their communi- 
cation skills. A well-intentioned 
effort to "make things easier" for a 
mentally retarded person by phrasing 
inquiries as yes-no questions. can 
have the effect of destroying that 
person's opportunity to communicate. 


In closing, we cite two of the less- 
structured interviews we conducted’ 
with mildly retarded,. relatively 
verbal adults living in the community, 
in which we asked them to reflect on 
the implications of being retarded, 
(virtually all of these adults denied 
being retarded, but had much to say 
about the implications of being con- 
sidered retarded.) We believe these 
transcripts are an excellent illus- 
‘tration of the value of asking re- > 
tarded persons to speak for themselves. 


r 


Interview I 
Q: What bad things happen to you 
“when people think you are re- 
tarded? 


A: Not too much has happened to me 
‘bad. They been pickin on me, 
They say, "You can't do that 
because you're mentally re- 
tarded. You can't do this." 
They just sort of push me back 
when I want to do something and 
say, "Oh, you can't come with 
us. We don't want you." + They 
just push you back into a cor- 
ner. "She can't do this! She 
doesn't know what she's doing." 


: What good things happen to you 
when people think you're re- 
targe? 


: Oh, they seem to want to take:. 
me out to ..: -erent places like 
football sez ‘+ and picnics 
and going out te uw falls and 
spending the day out there. 

And going on all day trips. 


Q: 
A: 
Q: 


Does anything else good happen? 
No. 


O.K., what would it be like ‘for 
you if people stopped thinking 
you were- retarded? 


: It'd be pretty good. We could 


probably get along a lot better 
if people didn't think of us 

as being retarded and handi- 
capped. We'd probably make it 
out on our own. But see, the 
rest of the people are just 
pushing you like in one certain 
square and saying, "That's 


` where you can be; you can't 


come over here." _ Because I've 
had that all my life. 


£ How do you think it would be 


different? What would you be 
doing differently if people 
hadn't done that to you? 


: Uh, I think I'd have higher 


paying jobs and be able to go. 
to restricted movies; not that 
we want to, but you know, if 
you wanted to, be able to do 
it and not having to say, "We 
don't allow you in here." 


When you say people telling you 
where to go, who do you mean 
exactly? 


: Generally everybody that thinks 


that... well, like when I 


“was going to school we had a 
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es; 


special section for special 
education. And everybody 

that went by there would call 
us names or throw things at us 
and say, "We don't want to be 
with you; you're mentally re- 
tarded. You're crazy." 
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A: Other people. The employees. 

: Bad. That's the ones that you have 

Q: What kinds of things do they do? a problem with when you jgo to 
work somewhere like that. The 

: Yell at you, like when I'm with employees. 
friends (unintelligible), Q: Are there any other things that 
things like that. Makes you : you'd like to do but can't 
feel about that big (gesturing because somebody thinks you're 
to show tiny size.) retarded? 

: Uh, huh. Does everybody do it, A: I don't know. They start making 
or does it seem like... l fun of you or something. Like 
I'm smart enough -to-know the yesterday _we.'were on a bus 
difference in people. Some of and this old lady was making 
‘em treat me real nice; like fun of my husband. And I didn't 
everybody else. Then some of like that at all. I was getting 
‘em don't. ready to tell her where to go! 

: Does it bother you when people Q: I imagine so. Just out of the 
think you are retarded? blue? 

: Yes, it does. I feel like I'm A: Yeah, out of the blue. Look 


: Yes. 


Interview Ii 


Q: 


How do most people treat you 
when they think you are retard- 
ed? S; 


that big (gestured small size). 
It scares me a little bit. 


like to do, but you cannot do 
them because somebody thinks 
you are retarded? 


‘ 


Q: What are those things? 


A: Well, like I've worked ina 


A: Treat me real bad. 


nursing home and they treat 
you real bad there. When they 
thought I was retarded. 


: Oh really? 


And they 
made me lose my.job just for 
that. f 


: How did they treat you? What 


did they do exactly? 


: I ‘tried to help them and they 


didn't want to be helped. And 


said I was retarded and all 


- that. . They were real ugly. 


: Are there things that you would `“ 


Were these the residents at the 
nursing home or the othe 
people who worked there? 


at you, and then make fun of 
you, and I don't think that's 
really right. It really does 
make me mad. It (unintelli- 
gible) you know, with our 
rights. You know we have our 
rights now. It's law we have 
our rights. If there's any- 
thing you want to, uh, through 
the law. I knew that long ago. 
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Interview Schedule: Community Children 


l. What is your full name? 
(Probe: "What is your last name?") 


2. How do you spell your name? 7 
(Probe: "How do you spell your last/first name?") 


3. Please write your name for me on this paper. 

4. How old are ycu? 

5. What is your address at home?- 

6. Are you usually happy or sad? 

7. How many friends do you have: a lot, some, not many, or none? 

8. When you're not at school, are you usually with other people? 

9. Do you play any sports? | 
10. . (If yes to "play spotter ack) What (other) sports do you play? 
ll. Are you usually happy? T 7 . 
12. Do you watch TV? 
13. Do yu listen to the radio or record player? 
14. Do you go out to eat at restaurants orj cafes? 
15. Do you go to church? > 


16. When you're not at school, are you usually by yourself or with other . 
people? : 


17. Do you know how to read books? 

18. Is anybody at school teaching you about reading now? 
19. Do you know how to write sentences? 

20. Is anybody at school teaching you about writing now? 
21. Do you know how to count money? 

22. Is anybody at school teaching you.about money now? 


23. Are you usually sad or happy? 
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24. When you're not at school, are you usually by yourself? 


25. Here are some ways people get to school. (Point as you say): They 
take the bus, somebody drives them to school in a car, they walk, or 
they ride to school on their bicycles. Which way do you get to school 
most days? Point to the picture. 

(Interviewer may probe with "Most days, how do you get to school?") 


26. Do you live in a house, an apartment building, a trailer house, or a 
duplex? 


27. Are you usually sad? 


28. Counting you, how many people live in. your house right now? 
(Interviewer probe, if necessary, with " How many is that, counting you?" 
or answer "Yes" if asked if we want a number.) 


29. When you're not at school, are you usually with other people or by 
yoursel f? l 


30. If you had one wish, what would you wish for? 
3l. Most of the time, is it up to you to decide what time you go to bed? 
32. What time does school start every day? 


33. Most days, does somebody else tell you what to wear to school, or “do 
you decide? 


(Probe, if necessary, with" "Most days?") 
34. -Is anybody at school teaching you about cooking now? 


35. This picture (point) shows a boy/girl who is happy, and this one 
(point) shows a boy/girl who is sad. Which picture shows how you 
usually feel? Point to the picture. : 


36. Here are some pictures of different kinds of places people live in. 
(Point to each as you say) Here is a duplex, a house,a trailer house, 
and an apartment building. Which picture shows the kind of place you 
live in? Point to the picture. 


37. Most days, how do you get to school? 
(Interviewer probe, if necessary, with Most days.") 


38. What things would you really like to learn in school? 
(Probe one time only with: What other things would you really like 
to learn in school?) 


39. This picture (point) shows a boy/girl who is by himself/herself, and 
this one (point) shows a boy/girl who is with other people. When you're 
not at school, which boy/girl is most like you? Point to the picture. 


40. Is anybody at school teaching you about numbers now? 
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41. 


42. 


43. 


44. 


45. 


47. 


48. 


49. 


50. 


51. 


52. 
53. 
54. 


55. 


56. 


what time do you usually go to bed at night? 
(Probe, if necéssary, with: "Usually. ") 


Most days, is. it up to you to decide what to wear to school, or does 


' somebody else tell you what to wear? 


(Probe, if necessary, with: "Most days.") 


Are you allowed to go places by yourself, or does somebody always take 
you? 


This picture (point) shows a boy/girl who is sad, and this one (point) 
shows a boy/girl who is happy. Which picture shows how you usually 
feel? Point to the picture. i 


Most of the time, does somebody eise tell yov what time to go to bed? 
(Probe,' if necessary, with: "Most of the time.") 


Do you live ina duplex, a hausa; a trailer house, or an apartment 
building? 


(Probe, if necessary, with: "Which one: duplex, a house, a trailer 
house, or an apartment?") 


Is anybody at school teaching you about keeping house now? 


Most days, do you get to school on the bus, in a car, by walking, or 
riding a bicycle? , : 


_ (Probe, if necessary, with: "Most days." Also may have to probe: 


"Which one: the bus, a car, walking, or a. bicycle?") 
Is anybody at school teaching you about getting jobs now? i 


Are you ever allowed to go places by yourself without somebody taxing 
you? 


Here are some pictures of different kinds of chores some people do. I 
want you to tell me if you do any of these chores at home. 


Do you set the table? 
(Probe, if necessary, with: "Is that a lot,.some or not much?") 


Do you do dishes? 
Do you clean the floor? 


Do you dust furniture? 
Do you make beds? : 

(If yes:) Do you make beds a lot, some or’not much? 

(Probe, if necessary, with: "Is that a lot, some or not much?"). 


Do you pick ‘up stuff around the house? : 

(If yes:) Do you pick up stuff around the house a lot, some, or not 
much? 

(Probe, if necessary witha "Is that a lot, some, or not much?") 


57. 


58. 


59. 


60. 


6l. 


62. 


63. 


Do you cook on the stove? 


Do you make sandwiches? 


.Do you take out the trash? 


Do you work outside in the yard? 


Here are some pictures of different kinds of places people live in. 
(Point to each as you Say:) Here is a house, an apartment building, 
a trailer hovse, and a duplex. Which picture shaws the kind of place 
you live in?» Point to the picture. ` 


Are the neighbor kids very friendly, just so-so, cr not friendly to 
you? 

(Interviewer probe, of necessary, with: "Which one; very friendly, or 
so~so, or not friendly?") 


This picture (point) shows a boy/girl who is with other people, and 
this one (point) shows a boy/girl who is by himself/herself. When 
you're not at school, which boy/girl is most like you? Point to the 
picture. f p : 
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Interview Schedule: Institutionalized Adults, Form A 


16. 


17. 


18. 


19. 
20. 
21. 


22. 
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What is your full name? 


How do you spell your name? 


. Please write your name for me ọn this paper. (Use back of form) 


. What month is your birthday? 


What day is your birthday? 
What year were you born? 
What is your address here? 


What kind of place is this? 


Do you have a family? 


How often do you leave here (where you live) to go see your family? 
How often does anybody in your family come to see you? 

Are you usually- -happy or sad? - 

Are you allowed to go on dates here? 

(If no) Why not? 


How many friends do you have? 


What do you and your friends usually do ‘together? 
(Probe: “What else?" until client can think of no more.) 


What do you usually do for fun when you are by yourself? 
Are you usually by yourself or with other people? 


Tell me about the most fun thing you did lasc week. 


What did you do? 


Who was there? 


When did it happen? 


Where did it happen? 


> 
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23. Why did you do that? 
24. Who decides what chores you do? 
25. Who decides how you aera your money? 
26a. Do you watch TV? \ 
b. (If yes) Do you watch TV alot some, or not much? 


27. How much do you listen to the radio, or record player: A lot, some, not 
much, never? 


28. How much do you read booki, magazines, or newspapers? 
29a. Do you go out to the movies? 
b. (If yes) Do you go to the movies not much, some, a lot? 
30. How often do you go out to eat? 
31. How often do you go to church? 
32a. Do you go to stores? 
b. (If yes) Do you go to stores not much, some, or a lot? 
33. Do you date? 
34. (If no) Why not? 
35. Do y... play any sports, like baseball or football? 
36. (If yes to "play sports") Which ones? 
37. Do you play any games indoors? 
38. (If yes to "indoor games") Which ones? 
"39. Do you do any arts and crafts? ( 
40. (If yes) Which ones? 
41. Is anyhody teaching you arts and crafts now? 
42. Are you usually happy? 
43, How many people sleep in your bedroom? 
44a. Do people eee yell at you /or say mean idide ' 


b. Do people here take things away from you? 
Á 
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c. Do people here hit you? 

d. Do people here make you do things you don't want to do? 

e. Do people here bother you when you want to be alone? 

f. Do estes here make fun of you? 

ai. Be Denie ake do, anything else that bothers you? 

h. (If yes to "anything else that bothers you") What? 
45a. Do people here help you when you want help? 

b. Do people here teach you things you want, to learn? 

c. Do people here give you presents? 

d. Do people bere mig you? 

e. Do people here talk to you when you want to talk? 

f. Do people here say nice things a jon 

g. Do people here do anything else that is nee? 

h. (If yes to “anything else nice") What? 
46. hes you usually with other people? 

Now I want to ask you some questions about the rules here: 

47. Is it against the rules here to stay up late at night? 
48. Is it against he rules here to hit people? l 
49.. Is it against the rules here eo ai people ugly meds 
50. Is it against the rules here to leave here without asking? 
51. Are Jon usually sad? : 
52. Did you’ ever get penbenad, ieres 
53. (ne yes) What did you gee punished for? 
54. (If yes RE PEN get punished") How were you punished? 
55. ‘Who makes the rules here? 


56. Do. you help make any rules here? 


57. Which Pen Mea al id how you fee. about living here. Point to the picture. 
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58. Why do you feel that way? . \ 
59. How do you like the food here? Not at all, not much, some, a lot. 
60. Why do you det that way? 
61. How do you like the er here? A lot, some, not much, not at all. 
62. Why do you feel that way? l 
63. Did you ever live someplace else? 
64. Did you live in another place last month? 
65. Did you live in another place last year? 
66. Do vols want to live someplace else? 
67. (If yes) Why? 
68. (If no) Why no? 
69. Are you usually ate, other people or by yourself? 
70. Do you have any problems? 
7la. Ro you usually have problems finding men/women friends? (same sex) 
b. Do you usually have problems finding men/women friends? (opposite sex) 
c. Do lyou usually have problems knowing what to do when you get sick? 
© d. Do! you usually have problems with anything else? 
m (If yes) ihat else? 


72. Who do you talk to about your problems? 
(Probe: Who is that exactly?) 


73; Who helps you the most when you need help? 
(Probe: Who is that exactly?) 


74. if you had one wish, what ‘would you wish for? 

75. Why do you wish that? 

76. Do you know how to read and write? 

77. Is anybody ‘teaching you about reading and writing now? 
78. Do you know how to cook? 


x 


79, Is anybody teaching you about cooking now? 
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80. (I£ yes) Who ‘is teaching you about ¢ooking? 
(Probe: Where are you learning about cooking?) 


8i. Do you know how, to keep house? 
‘82. Is anybody teaching you about keeping \house now? 
83. Do you set the table? (show picture) 
84. Do you do dishes? (cha picture) 

85. Do you do laundry? «(show aoura 
86. Do you clean the Bisex? (show picture) 
87. Do you dust furniture? (show picture) 
88. -Do you make beds? (stow picture) 

89. Do you pick up stuff arawd the house/dorm? (ùse proper term) 
90. Do you cook on the stove? (show sigare 
91. Do you make sandwiches? (show picture) 
92. Do you take out the trash? (show picture) 
93. Who decides what chores to do? 

94. Are you usually sad or ape 

95. Do you get enough food to’ eat? 


‘96. Do you go to school or take classes? 






97. (If yes) "What are they teaching you there?" 
(Probe: "What else?" Until the client can think of no mbre.)° 


98, Do you know how to get a job?, 
99. Is anybody teaching you about ‘getting jobs now? 


100. (If yes) Who is teaching you about getting. jobs? 
(Probe: Where are you learning*about getting jobs?) 


101. Has anybody from vocational rehabilitation ever tried to help you? 
102. (If yes) How did they help you? 
103. Are you usually by yourself? 


104. Do you have a job you get paid for? ; 
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105. 


106. 


121. 


122. 


Is anybody teaching you about money now? `~ 


(If no): Have you ever had a job? 


Dò you want to have a job? 


(If yes to "Do you have a job you get paid for?): Where do you work? 


(If yes to "have a job"): What exactly do oude at your job? 
(If yes to "have a job"): Do you like your job? 

(If yes to "have a job"): Do you want to keep your job? 

Did you have a job last month? 

Did you have a job last year? 


Do you know how to count money? 


(If yes) Who is teaching you about money? 
(Probe: Where are you learning about money?) 


Where do you get your money? 

Do you get SSI? 

Do you decide how to spend your money? 
Could you buy some candy with $1.00? - 
Could you buy a new bicycle with $1.00? 
Could you buy a new TV with $10.00? 


Could you buy a record album with $10.00? 


A.10 


O. 
ERIC 


APPENDIX B 


Accessing Mentally Retarded Persons. 
for Interviewing Purposes 


ERIC 


Accessing Mentally Retarded Persons 


for Interviewing Purposes 


In order for any survey of a sample of the mentally retarded population 


to be carried out, attention to the regulations and guidelines governing par- 
Cc : g 


ticipation in research projects is necessary. As indicated in Chapters 3 and 


4, the procedures required in the various sites used in our interviewing pro- 
ject were different from one another. An interviewing project of this type 


depends on the- cooperation of agencies and facilities to which mentally re- 


` © Ss + 


‘tarded persons are known. It is entirely within the. discretion of those agen- 


cies and facilities to develop their own rules about. access to clients, or in- 


there aže certain guidelines that have gained common acceptance in practice, 
either because they are mandated or because EE been strongly wavedens 
by key organizations. As part of our project, we solicited information about 
such regulations and guidelines through a letter sent bo a variety of public 


rà 


and private. organizations and agencies involved with the developmentally dis- 
= a E : 
abled. The following summary is intended to suggest certain key procedures 
and issues which must be borne in mind. 
Assuming that a study has been planned and an intended population has 
been’ identified, the first step necessary is to obtain’ approval of the re- 


search proposal from appropriate review bodies constituted for that purpose. 


Institutions which receive federal research money are required to have insti- 


tutional human uses committees to screen all proposals and insure that the 


rights of participants are adequately protected. Although such committees 


were originally established to review projects to be funded by the Department 


` deed to prohibit certain types of research or any research at all. Nonetheless, 


of Health, Education, and Welfare, i they are screening the whole 


range of research projects involving human subjects, whether or not they 
\ t + ` 


are funded externally. b \ 
| 


Institutional human uses committeęs are primarily concerned with the 


i 
1 


protection of confidentiality of information, the secural of informed consent, 


1 


and the protection be cubjects from physical or psychologica risk. The HHS 
regulations regarding the Protection of] Human Subjects Titte 45, Part 46, 
revised April 1, 1977) are perhaps the most commonly followed regulations in 

the country because they must be adhergd to if institutions are to receive 

.HHS funding for research. In addition, studies of national scope require. 
review of materials by the office of Management and Budget. In our own project, 
for example, such a review would have been required if we had actually conducted 
5 national survey, but ee not necessary since we restricted our studies to 
Arkansas and Texas. : 

Where the cooperation oe aNPaciiity or agency is required, as it inevi- 
tably would be in PN research with retarded percents: these E P too 
have institutional review committees that must approve proposed research. of- 
ten more than one body is involved. In Texas, for example, a research project 
to be conducted at a state school for the mentally retarded must first be ap- 
proved by a local committee including representation from the community. It 
is then forwarded to a state level committee constituted by the state Depart- 
ment of Mental Health and Mental Retardation for approval. In the a of 
Texas, then, any research to be ona ceed Dy university staff involving resi- 
dents of the local state school would be approved: by three bodies: the univer- 
sity human uses committee, and the local ae eee MHMR review bodies. In the 
case of een tre studies, it is highly unlikely that the project would be 
judged to place subjects at risk, assuming that appropriate precautions are 
taken to insure that information provided by interviewees remains confidential 
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and that informed consent for participation is obtained. Nonetheless, the 


researcher must recognize that the process of obtaining approval can be 


N 


very time consuming, taking as long as six months or so in our experience 
locally. Increasingly, too, ioual agencies and facilities are E 
hesitant to permit research which has no potential banetit for participants 
or for the facility staff. The benefits of participating in an interviewing 
study eee remote (e.g., contributing to information which can be Mom to 
improve mental retardation services in the future). 

Several bodies besides DHEW have developed guidelines for research which 
may influence local facilities and agencies and of which the mental retarda- 
tion researcher should be aware. The American Association on Mental Deficiency 
Legislative and Social Issues Gomi ttes, for M nas developed a Consent 
Handbook (AAMD, 1977). The AAMD position is that consent is not required for 
the use of aggregate data (for Biante: from facility statistical reports), 
but would be required where individai subjécts are involved unless the pro- 
ject involved anonymous interviewing about issues which are not personal in 
nature, in which case the subject's consent can be considered implied by his 
or her participation. rormal or explicit consent is recommended when the in- 

..- formation sought mighc identify the respondent or augue else, when it is 


we 


likely that unsought information will accompany the release of sought infor- 
A regs and when the information is protected under the First Amendment . 
While the AAMD generally advocates compliance with DHEW guidelines for in- 
ne ee the AAMD handbook is quite helpful in its explanation of consent 
as having three elements: capacity, information, and voluntariness. From 
the AAMD viewpoint, while DHEW emphasized information, there isa need in re- 
search with retarded persons for equal emphasis on determining the subject's 


capacity to understand wnat participation involves and insuring that the con- 


sent provided is indeed voluntary. These issues are certainly less important 
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in an interviewing project that involves no risk than/“‘they are in biochemi- 
A ‘ ay « 7 f 
cal or experimental research, but the AAMD, quite rightly, emphasizes that 


special procedures might be necessary in view of the cognitive limitations < 


of mentally retarded persons. Clark, Kivitz, and Rosen (1978) have dis- 


cussed in detail the tendency of mentally retarded persons to comply with 


i 


sy . ; . l 
various requests imprudently (for example, to sign petitions without reading 


them or take pills without finding out whether they are harmful). Unfor- 


z 


tunately, there are limits to the ways in which an interviewing study can be 
explained to pokenetel interviewees. The soundest procedure ected give a 
very simple and concrete description, encourage the potential participant to 
ask questions, and carefully, perhaps by asking the question in a namben oe 
different ways, insure iat the individual indeed eee to participate and is 
not simply complying without EE A When a subject i8 a young child 
or has been declared incompetent, a substitute decision-maker (e.g., parent 
or guardian) can make the decision on the subject's behalf. 

It should be noted that several bodies have developed positions in 
response to past exploitation of certain vulnerable classes of subjects, | 
especially residents of institutions. The National Commission for the Protec- 
tion GE waren Subjects of Biomedical and Behavioral Research has stated that 
such groups should no longer be overburdened. by the demands of research and 
underbenefited by its results. Other groups, for example the American Bar ’ 
Association, have taken the ERR position that if research uses mentally 
retarded subjects it must relate to mental retardation, thereby taking a 
stand against using captive groups to conduct research whick might be done 
with any number of subject Eo A DHEW proposal of 1974 sought tzo limit 
research with mentally retarded subjects to topics most likely to benefit 
the mentally retarded, and the American Nurses Association guidelines concur. 


The Joint `Commission for the Accreditation of Hospitals (JCAH) recommends ` 
; "y 
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that research in institutions be explained to the staff and that the re- 


search results be implemented in the institution. In the most extreme case, 
the Willowbrook OEE E EEN a court ruled that, given a history sé 
abuses within the facility, no research upon members of the plantiff class 
would be permitted thereafter. This inoveaged emphasis upon direct benefit 


evolves from an awareness of the coercive nature of institutions and the im- 
paired decision-making capacities of the mentally retarded people who reside. 
in them. 

All documents on the use. of human subjects in research emphasize the 
importance of maintaining a research participant's confidentiality. In 
particular, the problems of safeguarding against obtaining unsought informa- 
tion and prompting unanticipated outcomes are warned against. The confiden- 
tiality issue first crops up in identifying subjects. An interviewing study 
would require that agenvies of facilities participate in identifying current 


\ 


| . 
and/or former clients. | \L though the regulations do not appear to be standard 
o š . 


tie a: Sa 
in this area, a hard-line position would be that only the agency or facility 
could make initial contact with potential subjects, and would then release 
to researchers names and other requested information for those who provide 


informed consent to participate. ‘Indeed this was the procedure required in 


one of our sites. However, other agencies or facilities may be willing to 


give researchers access to client rosters and files on the condition that 


they use the information only to solicit consent for participation in the 
ne It is iikety that agencies and facilities will become more re- 
luctant to allow this access in the future. rE the regulations in a 
given setting, the researcher does have a clear responsibility to keap the 


names of potential or actual participants strictly confidential. 
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Interviewing studies may give rise to unanticipated problems of 


confidentiality. For example, the APA (1973) research participation guide- 
lines cite the example of a study of deinstitutionalized mentally retarded 
persons. Risks may ayia if such a study Giiees contacts with neighbors, 
paige ana employers, wno may not have previously realized that the 
subject. was mentally retarded or institutionalized. The precaution recom- 
mended by the APA is to restrict interviews to staff who already know the 
subject's status and to obtain permission from the sabet if anyone besides 
the subject is to be interviewed. There is still the risk, however, that a 
mentally retarded person might not fully understand the implications of 
having his or her employer or neighbors’ interviewed. - 

Another issue, also described in the APA document, arises whén the re-_ 
searcher happens to acquire sensitive information, arten information unre- 
lated to the research itself. Examples especially relevant to interview 
studies include the discovery that a research subject abuses drugs, carries 
a weapon, is suicidal, or is nadin in unhealthy or destructive behavior 
of some sort. In such cases, researchers face the options of disclosing the 
information and thereby violating the subject's confidentiality or not dis- 
ee it and possibly being responsible, in a moral if not legal sense, 
for subsequent~harm to the subject or others. Responsibilities of the re- 
searcher to disclose or not to disclose vary from state to state, and quite 
obviously the ethical issues are complex. One potential solution is to in- 
clude in informed consent statements an indication that the researchers might 
violate confidentiality if they feel a subject's welfare is endangered. in 
our own Center's research, a less severe but still difficult problem has at 
times arisen. Deinstitutionalized persons who have been interviewed have on 


occasion had transparent needs for help from service deliverers or have actually 


asked the interviewer to obtain help. Here, while it has been our policy to 
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comply with requests by yeferring the individual to appropriate local agen- 

| cies, the research E E T has introduced a change in the life being 
studied. Otherwise, confidentiality has been strictly iain: Indeed, 
clients had to be carefully convinced that participating would in no way 
jeopardize their community placements. 

Of course researchers always have a responsibility to protect the in- 
formation they collect and avoid giving it to other Parties unless subjects' 
explicit. permission has been obtained. Subjects must not be identified or 
identifisbis in research reports or other dissemination vehicles unless they 
have given specific consent. 

In summary, our conclusion is that regulations and guidelines for re- 

- search do not pose insurmountable problems in an interviewing study, but 

are problematic. If a national study were to be conducted, problems would 

be compounded. Obtaining permission to conduct the research from a variety 
of agencies and facilities in a variety of settings is likely to be time- 
consuming and costly and certainly must be considered in project funding ae 
staffing. In an interviewing study, Hie ds unlikely to be an betae to 
approval; but in Sone settings questions may be raised about the Likely bene- 
fit to paveleipanee and participating facilities, aha in all probability the 
researchers will have obligations to provide reports to participating facili- 
aies: The most difficult problem, we believe, is identifying eligible sub- 
jects initially. -Where explanation: letters and consent forms must be mailed 
by various facilities or agencies before ianea can be released to resėarchers, 
one may have difficulty obtaining agency cooperation because of the time dm 
dived in mailings and a low rate of returns is likely, making the representa- 
tiveness of the final sample suspect. Efforts to obtain consent which involve 
personal contact, by phone or better yet face-to-face, are likely to yield 


far higher rates of return, but may still be prohibitively time-consuming 
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from an agency perspective if agency staff must make the contacts. The, 
process is sasiest if clients are physically located in a facility, more 
difficult if they cnly visit an attiée, and most difficult if former Six 
ents are to >e included in the dumpis, doz then the initial task of loca- 
ting them may prove challenging and costly. By comparison, thu tavs « 
obtaining consent and maintaining confidentiality once the project is 
underway are quite manageable. | 

The following materials may prove useful to researchers planning sur- 
vey research, or any type of research, with mentally retarded populations:, 


American Association on Mental Deficiency: Consent Handbook. Washington, 


D.C.: AAMD, 1977 (Also statement on the use: of human subjects for l 


research). 


American Psychological Association. Ethical principles in the conduct of 


research with human participants. Washington, D.C.: APA, 1973. 


National Association for Retarded Citizens: Mental retardation research: 
Guidelines for the use of behavior procedures in state programs for 


retarded persons, and Guidelines for biomedical and pharmacological 


research procedures and the protection of human subjects in residential 


facilities for. mentally retarded persons. 


National Commission. for the Protection of Human Subjects of Biomedical and 


Behavioral Research: Protection of human subjects: Disclosure of re- 


search information under the Freedom of Information Act; and Research 


with institutionalized mentally infirm subjects: Recommendations. 
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U.S. Department of Health, Education, and Welfare: A variety of reports and 
regulations, especially Code of Federal Regulations, Public Welfare 


45CFR 46, Protection of Human Subjects, 1977, revised edition. 
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Texas Tech University 


Research and Training Center in Mental Retardati> ‘ (806) 742-3131 
Training Division / (806) 742-2974 


’ 


i 


Dear Parent: 


The President's Committee on Mental Retardation** want to i 
the best ways to get information from and about retarded pe .e. 
have asked the Research and Training Center in Mental Reta: tior 
Texas Tech to find this out for them. The Research and T: -ning v 
would like Lubbock students to help with this project. We would like 
to talk to some students to see if they can understand and respond to 
the questions we have developed, and we must have your permission before 
we can do this. 
The Research and Training Center will randomly pick about 60 students 
from those whose parents give their consent. Then, two trained interviewers 
will spend about 15 minutes during the school day talking to each student 
who was chosen. This is not a test: it is only an interview. The interview 
will not change your child's school programs or placement in any way. 


The students will be asked some y ‘estions like "What is your birthday?" 
to see if they can give accurate infornation about themselves. They will 
be asked about their feelings about themselves, others, and their situations. 
For example, one question will be “Are you usually happy or sad?" They will 
also be asked to tell about activities they are involved in like sports, 
indoor games, arts and crafts, and chores. some of the questions ask about 
services they are getting and people who help them. (For example: "Is any- 


body teaching you to read now?") Finally, there will be some questions about 
rules in the home and school and decision making. (These will be questions 
like: “Are you ever allowed to go places by yourself without someone taking 
you?") s f 


All of this-is completely anonymous. Everything your child tells the 
interviewers will be secret. We will not éven tell anyone at the schoòl 
what his/her answers are. In fact, the Research and Training Center is 
not really interested in what your child's answers are; we mainly want to 
know how well your child understands our questions. Also, if there are 
questions your child does not want to answer, he/she does not have t2. 

If the interviewers are told or get the feeling that a student does not 


want to finish the interview, they will stop. 


There may be some students who will not be able to communicate very well 
or at all. We still need permission for them to be part of the study because 
one thing the President:'s Committee wants to know is how many children cannot 
respond’ at all to the questions. ' : 


**The President's Committee on Mental Retardation is a committee of 
people appointed by the President of the United States who help the Presi- 
dent and Congress make policies and laws for retarded citizens of our coun- 
try. i 
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Besides the student interviews, the Research and Training Center would 
like to spend’about 20 thinutes interviewing a parent or someone else who 
lives with the student and knows him/her’v czy well. We will be asking you 
questions about your child to find out if the answers your child gave were 
accurate. For example, if a student «-."wered that he sets the table "a 
lot, Would the parent also „šav that tki} child sets the table "a lot,” 
or would he/she say "not. mach?” For this parent interview, the interviewers 
will contact you before they come to your home or any place you would like at 
a time that would be best for you; even nights or weekends if necessary. 
There is a place for your phone number and address on the consent form. 
Please fill this in so they can contact you to schedule the parent interview. 


Finally, the Research and Training Center would like to have your per- 
mission to use some basic information about your child from his/her school 
records. We would like the school to tell us your child's birthday, IQ 
score, and if he/she has other handicaps. 


If you and your child help with this project, the Research and Training 
Center will be glad to give you a copy of the. results. While this report will 
not tell you anything about your child individually, .* will give you.some 
information about how children feel about their lives and what their strengths 
and weaknesses in communication skills are asia group. 


ff you have any questions about this project, please feel-free to,call 
Carol Schoenrock at the Research and Training Center. Her phone number is 
742-3134 or 742-3131, extension #38, She will, be happy to answer any ques- 
tions you might have. about this study. y f 


When you have decided whether or not you Wane your chta to help with 
this project, please fill out the attached form, showing/that ’ you either 
give your permission or do not give your permission £65 your child to partici- 
pate. Please give this form to your child to take back to school tomorrow. 
Thank you for your consideration. 


$ 
I 
s e | Sincerely, . 


Coat K: Liqa 


Carol K. Sigelman, Ph.D. 
Director of Research 


n 











Texas Tech University 


Research and Training Center in Mental Retardation / (806) 742-3131.. 
Training Division / (806) 742-2970 i 


August 7, 1978 
Dear Advccate: 


The President's Committee on Mental:Retardation has asked the Research and 
Training Center to undertake a project for them. They want us to determine 
tne feasibility of establishing a national polling system that would enable. 
them to quickly gain inforz..tion about the needs, circumstances and atti- 
tudes of retarded citizens. By getting input from mentally retarded 
consumers themselves, they hope to make national policy formation more 
sensitive to the actual needs and opinions of these consumers. 


i 


As part of this study, we are presently trying to determine the best meth- 
ods for obtaining certain types of information from ard about retarded per- 
sons; in this case, adults living in the community. The Citizen Advocacy 
office in Austin is helping us to establish a sample of such persons to 
interview, and your name was given to us because you are an advocate of . 
someone we want to include in that sample. 


During the last few weeks in August, a trained interviewer from the Center 
will conduct personal interviews with the proteges. These interviests should 
take approximately 30 minutes each and’will focus on residential living 
circumstances and on services received. We will address ourselves to the 
extent of the proteges' knowledge about these areas as well as their atti- 
tudes toward them. In addition to this, a telephone interview will be 
conducted with each person's advocate. (A personal interview can be ar- 
ranged if the advocate desires.). This 15-minute interview will cover the 
same topics found in the protege intervi-w, allowing us not only to deter- 
mine the accuracy of the protege's responses to different types of ques- 
tions, but also to obtain a different perspective. Of course, both inter- 
views ‘will be completely anonymous and confidential. i 

\ 
Before ‘we can interview you or your protege, we need written, informed con- 
sent fram both of you. Enclosed with this letter is an explanation of our 
project which we ask that you read to and discuss with your protege before 
he/she Signs the consent form (also enclosed.) If your protege is not 
capable of giving informed consent, please discuss this with his/her guard- 
ian and lèt him/her sign the concent form in the space labeled GUARDIAN 
CONSENT. Regardless of who signs the consent form, we would appreciate 
your informing the guardian (or other person acting as caretaker unless the 
protege lives independently) that we will be contacting them or your pro- 
tege to arrange a time and place to conduct the interview. 


We have enclosed a self-addressed, stamped envelope for your- convenience in 
returning the consent form. This needs to be returned whether both of you 

give your consent or not. The President's Committee also wants information 
about refusal rates. However, we hope you will help with this project, for 
we can all benefit from techniques to help retarded citizens express their 

points of view. If you or anyone else have any questions, I would be more 

than happy to accept a collect call at (806) 742-3131. 


Sincerely, 


Carol Jean Schoenrock 
Project Coordinator 
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2. 


3 


EXPLANATION OF R & T CENTER: PROJECT 


(To be read to the protege by the advocate) 


Some people who work at Texas Tech University in Lubbock want to come 
and talk to lots of people who live in Austin. If you want to, they 
would like to talk to you for about 10 minutes. 


They want to ask you questions about where you live, what places you 

go to get help or learn things, who is teaching you things, what things 
you and I do together, and how you feel about where you live and the 
help you get. i 

They want to find out the best way to ask people these’ questions, and 
they want to find out what things are like for people who live in Austin. 


4. No matter what you tell them it will, not mak- aay changes in your life. 


5. 


6. 


7. 


8. 


| They are not trying to test or evaluate you; they want to test their 
questions to see if the questions are good enough to use with people 
an over the United States. 


You do not have to talk to them at all if you do not want to. If you 

do talk to them, everything you say will be a secret. Nobody else 
except the person you talk to will knew exactly what you say. They won't 
even tell me what you say. If you talk to them and there is a question 
you don't want to answer, you don't have to; you can just tell them you 
don't want to answer that. 


They also want to talk to me and ask me some questions about you and the 
things I do with you and for you. Everything I say will be a secret, 
too. They won't tell anybody else, even you, what I tell them. 


Before they interview you or me, they want to look at ne rc 
Citizen Advocacy office to get some basic inforitation. 

They will senda report to the Citizen Advocacy office and to some other 
people-about the interviews they do with. everyone here in Austin. But 


_- tt will not say anything about you or me. It will say things like: 


9. 


10. 


"Such ‘and such was a good question; most people understood it," or 
"This was not a good question to ask; we need to find a better way to 
ask it." Also, they will say things in the report like: "Most pro- 
teges said their advocate did such and such with them," and "Many of 
the proteges would like to learn such and such." 


Before they interview you, tHey will call or come by your house to set 
J Up a time to do the interview. That way, you can tell them the best 

| time and place for you to do it sz they won't bother you when you are 
| going to be busy. 
If you have any questions about this project that I can't answer, we can 
call the people at Texas Tech and ask them before you decide if you want 
to help them. = 
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11l. When you decide, you need to sign the consent form and check one of the 


blanks telling them that you do want to help or that you do not want to 
help. Then I will send it back to them. If you check that you do want to 


help, they will get in touch with you in a couple of weeks to do the inter- 
view. 
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ADVOCATE'S NAME: PROTEGE'S NAME: 





CONSENT FORM 


I understand the project that the Research and Training “enter is doing. I un- 
derstand that I will be.interviewed and that what I sav «ill be secret and anon- ™” 
ymous. I know that I do not have to answer any questions I do not want to an- 
swer and that this interview will not mean any changes in my situation. I also 
understand that the interviewer will use information from my records at the Citi- 
zen Advocacy office. 


PROTEGE : 
(check one) 
YES, I do want to talk to the people f:om Texas Tech. 
No, I do not want to.talk to the people from Texas Tech. 
PROTEGE'S SIGNATURE: ` ` DATE: 
GUARDIAN CONSENT : DATE: 
ADVOCATE: 


(check one) 


YES, I do want to talk to the people from Texas Tech. 


No, I do not want to talk to the people from Texas Tech. 


ADVOCATE'S SIGNATURE DATE: 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 


IF YOU HAVE AGREED TO HELP WITH THIS PROGECT, PLEASE COMPLETE THE FOLLOWING INFOR- 
MATION: PROTEGE: 


Age: ; Sex: Telephone number: 





Address (include town & zip code): 





If we should contact someone other than the protege to set up the ap- 
pointment for the interview, please give us their name (plus phone and 
address if different from the Protege's): 








A 
N 














ADVOCATE: 
Te aone number: 





Address (include town & zip code): 








If there is any time that is generally better than another for us to 
contact you or your protege, přease indicate below: 


Advocate: Protege: i 
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SAMPLE INTERVIEW SCHEDULE PAGE 


The following page is a reproduction of the first page of the 


interview schedule used with community children. Regarding the method 


used to code answers, the "C" notation refers to content and is where the 


respondent's answer is coded. Multiple C columns were used for open-ended 
questions in which several answers could be enumerated. The "R" variable 
for each question is the responsiveness code for that question. If the 
interviewee = unable to respond adequately to the question - for example 
if his answer was irrelevant - the R code would indicate that and the C code 
would designate missing data. This system made it easy to keypunch data 


directly from the interview schedule. 
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Coding System Used on mac gn 


HS C: . os: 
l. ARE YOU USUALLY HAPPY OR SAD? f 
Sad = 1 Happy = 2 T HS1 R: 
2. ARE YOU USUALLY WITH OTHER PEOPLE? i WOP Cir 
No = 1 Yes Fa 
/ WOP1 R: 
vA 
f 


Ka 
The blanks in the right-hand column permit keypunching direct- 
ly from the actual interview schedules. The respondent answered 
the first question "sag". The "C" in the right-hand column indi- 
cates content sodun and the "sad" res}^nse is accordingly coded. 
l. The "R" indicates a responsiveness code; the 8 represents a 


minimally appropriate response. The numbers beneath these coding 


blanks indicate the correct column on the cards used in keypunching. 
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Been os An os - TRCOR recta =p Partial 292° correct = yp 


CLIENT INTERVIEW g Client Name: 
Consumer Study 


Interviewer Name: 


f Date of Interview: 
Research, and Training Center i 


in Mental Retardation Time Scheduled: 


Beginning Time: 
Texas Tech University 
Lubbock, Texas Ending Time: 


——__—_—_—=~—=—__—_ aaaea 


COMMUNITY CHILDREN 


0: a3 
STUDY 1 2 
CARD ae a 
3 4 
CASE TOE RE 
5 6 7 8 
INTNUM oo, 
—_— 


l1. WHAT IS YOUR FULL NAME? 


(Probe: “What is your last name?" Inte viewer may an~ C: : 
swer questions: no middle name needed, real name; 11 
not nickname, etc.) 


Incorrect = 1 ‘Partial = 2 Correct = 3 
g 
a ee 
2. HOW DO YOU SPELL ZOUR NAME? 
(Probe: "How do you spell your last/first name?") C i 


Int. may answer questions:' first and last name, 13 
not’ mičðic name, real name, etc.) 








-= 





3. PLEASE WRITE YOUR NAME iR ME ON 7:45 PAJER. 


(se .ack of form. Int. may anSwe. | 2s .oms: first c 
and last name, not middle name, real name, etc.) 


Incorrect = 1 Partial = 2 Correct = 3 R 





CLIENT CONSENT AND INTRODUCTION 


HI, MY NAME IS . I'M TRYING TO FIND OUT THE 





BEST WAY TO ASK PEOPLE QUESTIONS. A LOT ^F PEOPLE’ HERE. IN LUBBOCK ARE GOING“ 
TO HELP ME, AND I'D LIKE FOR YOU TO HELP ME, TOO. YOU DON'T HAVE TO DO THIS 
IF YOU DON'T WANT TO, JUST LET ME TELL YOU WHAT I WANT YOU TO DO, AND THEN 
YOU CAN TELL ME IF YOU WANT TO HELP. ‘I WANT TO ASK YOU SOME QUESTIONS ABOUT 
YOURSELF; MOSTLY ABOUT HOW YOU FEEL AND WHAT YOU DO AT HOME AND AT SCHOOL. 

IT WILL TAKE ABOUT 15 MINUTES. EVERYTHING YOU TELL ME WILL BE A SECRET. I 
WON'T TELL ANYBODY ELSE WHAT YOU. SAY. IF YOU DON'T WANT TO ANSWER A QUESTION, 
YOU DON'T HAVE TO. JUST TELL ME YOU DON'T WANT TO ANSWER IT. THIS 15 NOT A 


TEST. ANY ANSWER YOU GIVE MF IS OK. I'M JUST TRYING TO FIND OUT IF PEOPLE 


CAN UNDERSTAND MY QUESTIONS. DO YOU WANT TO HELP ME DO THIS? 


(If no, stop here. Thank client for taking time to listen.) 


(IF yes, continue with explanation below.) 


OK, GOOD. BEFORE WE START, LET ME TELL YOU SOME THINGS. SOME OF THE QUESTIONS 
MAY SOUND SILLY TO YOU, BUT I HAVE TO ASK THEM ALL, EVEN IF I ALREADY KNOW 
THE ANSWER, I STILL HAVE TO ASK IT. I'M GOING TO BE ASKING A LOT OF THE QUES~ 


TIONS MORE THAN ONCE. THAT WON'T MEAN THAT YOU GAVE ME THE WRONG ANSWER THE 


Tr 


K 
T7 FIRST TIME: “THE ONLY “REASON I'LL ASK SOME OF “THE QUESTIONS” MORE THAN ONCE IS” ~ 


TO FIND ONT WHICH WAY OF ASKING THEM IS THE BEST WAY. REMEMBER, TELL ME IF 


YOU DOM Be ^ ANSWER A QUESTION I'' d : Ue 
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